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Since  my  confirmation  as  Director  of  Operational  Test  and  Evaluation  (DOT&E)  in  2009, 1  have  implemented  initiatives  to 
improve  the  quality  of  test  and  evaluation  (T&E)  within  the  Department  of  Defense.  I  have  emphasized  early  engagement 
of  testers  in  the  requirements  process,  improving  system  suitability  by  designing  reliability  into  systems  from  the  outset, 
and  integrating  developmental,  operational,  and  live  fire  testing.  Implementing  these  initiatives  has  revealed  the  need  for  an 
additional  area  of  focus  -  the  requirement  to  incorporate  statistical  rigor  in  planning,  executing,  and  evaluating  the  results  of 
testing. 

There  are  significant  opportunities  to  improve  the  efficiency  and  the  outcomes  of  testing  by  increasing  interactions  between 
the  testing  and  requirements  communities.  In  particular,  there  should  be  early  focus  on  the  development  of  operationally 
relevant,  technically  feasible,  and  testable  requirements.  In  this  Introduction,  I  discuss  the  crucial  role  the  T&E  community 
can  and  should  play  as  requirements  are  developed.  Additionally,  I  describe  DOT&E  efforts  to  institutionalize  the  use  of 
statistical  rigor  as  part  of  determining  requirements  and  in  T&E.  I  also  provide  an  update  on  the  Department’s  efforts  to 
implement  reliability  growth  planning  and  improve  the  reliability  and  overall  suitability  of  our  weapon  systems.  And  lastly,  I 
describe  challenges  and  new  developments  in  the  area  of  software  T&E. 

Last  year,  I  added  a  new  section  to  my  Annual  Report  assessing  systems  under  my  oversight  in  2010  -  2011  with  regard 
to  problem  discovery  during  testing.  My  assessment  fell  into  two  categories:  systems  with  significant  issues  observed  in 
operational  testing  that  should,  in  my  view,  have  been  discovered  and  resolved  prior  to  the  commencement  of  operational 
testing,  and  systems  with  significant  issues  observed  during  early  testing  that,  if  not  corrected,  could  adversely  affect  my 
evaluation  of  those  systems’  effectiveness,  suitability,  and  survivability  during  Initial  Operational  Test  and  Evaluation 
(IOT&E).  This  year,  I  am  providing  an  update  to  the  status  of  those  systems  identified  last  year,  as  well  as  my  assessment  of 
systems  under  my  oversight  in  2012  within  those  two  categories. 


THE  ROLE  OF  T&E  IN  REQUIREMENTS 

There  is  an  inherent  and  necessary  link  between  the  requirements  and  the  test  communities.  The  requirements  community 
must  state  our  fighting  force’s  needs  in  the  form  of  concrete,  discrete  capabilities  or  requirements.  The  testing  community 
must  then  assess  a  system  that  is  developed  and  produced  to  meet  those  requirements  to  determine  whether  it  provides  the 
military  capability  being  sought;  that  is,  we  evaluate  the  system’s  operational  effectiveness  and  suitability  when  used  by  our 
forces  in  combat.  In  my  opinion,  the  collaboration  needed  between  the  requirements  and  the  test  communities  to  discharge 
these  responsibilities  needs  to  be  strengthened. 

In  my  report  last  year,  I  discussed  the  Defense  Acquisition  Executive  (DAE)  independent  assessment  of  concerns  that  the 
Department’s  developmental  and  operational  test  communities’  approach  to  testing  drives  undue  requirements,  excessive  cost, 
and  added  schedule  into  programs.  The  DAE  assessment  team  “found  no  significant  evidence  that  the  testing  community 
typically  drives  unplanned  requirements,  cost,  or  schedule  into  programs.”  However,  they  did  note  that  there  were  four 
specific  areas  that  needed  attention: 

“ The  need  for  closer  coordination  and  cooperation  among  the  requirements,  acquisition,  and  testing  communities; 
the  need  for  well-defined  testable  requirements;  the  alignment  of  acquisition  strategies  and  test  plans;  and  the  need 
to  manage  the  tension  between  the  communities.  ” 

The  lack  of  critically  needed  collaboration  among  the  technical,  test,  and  requirements  communities  is  not  new.  The  1986 
Packard  Commission  found  that  success  in  new  programs  depends  on  “an  informed  trade-off  between  user  requirements,  on 
one  hand,  and  schedule  and  cost,  on  the  other.”  It  therefore  recommended  creation  of  a  new  body  representing  both  military 
users  and  acquisition/technology  experts.  This  ultimately  led  to  the  creation  of  the  Joint  Requirements  Oversight  Council 
(JROC),  which  includes  the  military  operators  as  formal  members  but  includes,  as  advisors  only,  the  acquisition  and  test 
communities.  In  1998,  the  National  Research  Council  (NRC)  identified  the  need  for  greater  interaction  between  the  test  and 
the  requirements  communities;  the  NRC  pointed  out  that  operational  test  personnel  should  be  included  in  the  requirements 
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process  in  order  to  assist  in  establishing  “verifiable,  quantifiable,  and  meaningful  operational  requirements.”  And  the 
National  Defense  Authorization  Act  for  FY 1 1  specifically  named  DOT&E  as  an  advisor  to  the  JROC.  However,  obstacles 
for  close  collaboration  remain.  I  discuss  below  three  specific  areas  where  increased  interactions  could  result  in  improved  test 
outcomes,  which  should  then  result  in  systems  with  needed  and  useful  combat  capability  being  delivered  to  our  forces  more 
quickly. 

Mission-Oriented  Metrics 

OT&E  is  defined  in  Title  10  United  States  Code  as: 

“The  field  test,  under  realistic  combat  conditions,  of  any  item  of  (or  key  component  of  weapons,  equipment,  or 
munitions  for  use  in  combat  by  typical  military  users;  and  the  evaluation  of  the  results  of  such  tests.  ” 

Weapon  systems  sit  in  the  motor  pool,  at  the  pier,  or  on  the  runway.  Individual  systems  do  not  have  missions;  it  takes 
Soldiers,  Sailors,  Airmen,  and  Marines  to  make  them  work.  Operational  testing  is  about  assessing  mission  accomplishment 
of  the  unit  equipped  with  a  system.  To  evaluate  operational  effectiveness  we  seek  to  answer  the  question,  “can  a  unit 
equipped  with  the  system  accomplish  the  mission?”  Operational  effectiveness  is  defined  in  the  Joint  Capabilities  Integration 
and  Development  System  (JCIDS)  manual  as: 

“Measure  of  the  overall  ability  of  a  system  to  accomplish  a  mission  when  used  by  representative  personnel 
in  the  environment  planned  or  expected  for  operational  employment  of  the  system  considering  organization, 
doctrine,  tactics,  supportability,  survivability,  vulnerability,  and  threat.  ” 

And  the  Defense  Acquisition  Guide  emphasizes  “the  evaluation  of  operational  effectiveness  is  linked  to  mission 
accomplishment.”  End-to-end  testing  with  operational  users  across  the  intended  operational  envelope  is  essential  to 
assessing  the  system’s  impact  on  mission  accomplishment.  Additionally,  each  system  must  be  evaluated  within  the  context 
of  the  system-of-systems  within  which  it  will  operate. 

In  January  2010, 1  provided  guidance  to  the  Operational  Test  Agencies  on  the  reporting  of  OT&E  results  reiterating  that 
the  appropriate  environment  for  any  operational  evaluation  includes  the  system  being  tested  and  all  interrelated  systems 
needed  to  accomplish  an  end-to-end  mission  in  combat.  I  emphasized  that  the  primary  purpose  of  OT&E  is  to  describe 
the  operational  effectiveness  and  suitability  of  the  system  being  tested  within  that  environment.  A  subsidiary  purpose  of 
OT&E,  stated  in  DoDI  5000.02,  is  to  determine  if  thresholds  in  the  approved  Capability  Production  Document  (CPD)  have 
been  satisfied.  The  measures  used  for  this  purpose  are  appropriately  referred  to  in  the  context  of  “performance”  as  in  “key 
performance  parameters  (KPPs),”  or  “measures  of  performance.”  But  these  measures  associated  strictly  with  evaluating 
KPPs  are  not  the  full  set  necessary  to  evaluate  operational  effectiveness  in  combat. 

Requirements  are  often  stated  in  terms  of  technical  parameters  whose  satisfaction  is  necessary,  but  not  sufficient  to  determine 
a  system’s  effectiveness,  suitability,  and  survivability  when  used  in  combat.  Ideally,  KPPs  should  provide  a  measure 
of  mission  accomplishment,  lend  themselves  to  good  test  design,  and  encapsulate  the  reasons  for  procuring  the  system. 
However,  DOT&E  has  seen  many  examples  of  KPPs  that  are  not  informative  to  an  evaluation  of  mission  accomplishment. 
For  example,  a  previous  ground  combat  vehicle  had  KPPs  that  only  required  it  seat  nine  passengers,  be  transportable  by  a 
C-130,  and  have  a  specific  radio  system;  these  requirements  could  have  been  met  by  a  passenger  van.  Another  example  was 
an  amphibious  ship  with  KPPs  for  the  number  of  helicopter  spots,  the  number  of  storage  spaces,  and  the  maximum  speed  of 
the  ship;  these  requirements  could  have  been  measured  with  a  stopwatch  and  a  tape  measure  and  could  have  been  satisfied 
by  a  commercial  ship  with  no  capability  to  survive  amphibious  combat.  While  these  technical  performance  requirements 
are  important,  they  are  not  sufficient  to  determine  whether  the  ground  vehicle  or  ship  can  be  used  successfully  in  combat. 

In  these  cases,  the  test  community  encouraged  the  use  of  metrics  for  evaluation  directly  tied  to  mission  success  such  as 
accomplishing  geographic  objectives  while  minimizing  blue  force  losses  or  meeting  an  aircraft  sortie  generation  rate  and 
surviving  likely  attacks.  If  the  test  and  requirements  communities  engage  early,  requirements  can  be  stated  in  a  manner  that 
makes  them  directly  relevant  to  mission  success  and  therefore,  both  directly  relevant  to  operational  testing  and  much  more 
capable  than  technically-oriented  parameters  of  informing  whether  the  sought-for  combat  capabilities  have  been  achieved  in 
the  system  to  be  produced. 

Leveraging  T&E  Knowledge  in  Setting  Requirements 

Interactions  between  the  requirements  writers  and  the  testers  can  also  help  identify  alternatives  to  hard-to-test  or  impossible- 
to-test  requirements.  Requirements  that  cannot  be  verified  in  testing  may  as  well  not  exist.  The  T&E  community  can 
help  identify  unrealistic,  unaffordable,  and  un-testable  requirements.  Additionally,  T&E  knowledge  of  the  current  threat 
environment  and  test  infrastructure  can  help  the  requirements  community  understand  what  resources  will  be  needed  to  test 
a  given  requirement.  We  have  seen  Service  requirements  officers  state  they  want  demanding  if  not  technically  unachievable 
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requirements  to  drive  vendors  to  deliver  the  best  possible  system  performance;  but,  history  has  shown  setting  very  high  or 
unachievable  requirements  is  particularly  destructive  to  program  success.  For  example,  the  Future  Combat  System  program 
required  high  survivability  (“tank-like”)  and  tactical  transportability  (via  C-130)  that,  together,  were  impossible  to  achieve. 
Additionally,  reliability  requirements  for  that  system  were  much  higher  -  nearly  10  times  -  that  of  our  current  systems, 
making  achievement  of  those  requirements  both  unrealistic  and  unaffordable.  Clearly,  we  should  not  eliminate  requirements 
simply  because  they  are  difficult  to  test.  We  must,  however,  carefully  consider  whether  difficulty  (or  impossibility)  of  testing 
requirements  implies  the  same  for  their  achievement. 

Testers  have  experience  with  the  difficulty  and  cost  associated  with  the  testing  needed  to  demonstrate  whether  certain  metrics 
have  been  achieved.  For  example,  consider  a  requirement  for  99  percent  reliability  for  completing  a  6-hour  mission.  This  is 
comparable  to  600  hours  between  failures  and  would  require  at  a  minimum  1,800  hours  of  testing  to  verily.  However,  if  the 
requirement  were  95  percent  reliability  for  completing  the  same  6-hour  mission,  the  associated  mean  time  between  failures 
is  only  120  hours  and  testing  to  that  requirement  could  be  accomplished  in  a  minimum  of  360  hours.  If  the  testing  revealed 
40  hours  between  failures  (instead  of  120  or  600)  that  would  indicate  an  86  percent  probability  of  completing  a  6-hour 
mission.  Would  95  percent  or  86  percent  be  good  enough?  To  answer  that  question,  the  rationale,  or  so-what  factor,  for 
the  requirement  should  be  fully  explained.  Accordingly,  1  intend  to  require  that  Test  and  Evaluation  Master  Plans  (TEMPs) 
have  an  annex  explaining  the  user’s  rationale  for  the  requirements  contained  in  the  Capability  Development  Document.  The 
requirements  and  their  associated  rationale  should  be  revisited  as  often  as  needed  as  a  program  proceeds  and  knowledge  is 
gained  regarding  the  ability  to  achieve  the  program’s  currently  stated  requirements. 

In  addition  to  the  value  selected  for  a  requirement,  the  manner  in  which  a  requirement  is  stated  can  also  make  testing 
expensive  or  impractical.  For  example,  metrics  stated  as  binomial  probabilities  (99  percent  probability  of  detecting  a  target) 
are  expensive  to  test  because  they  require  large  sample  sizes  to  gain  statistical  confidence  in  the  results.  Metrics  that  are 
physical,  continuous,  easily  measured,  and  operationally  meaningful  can  be  used  instead  of  such  probabilities.  For  example, 
the  “median  miss  distance”  can  be  measured  at  high  confidence  with  about  a  third  the  number  of  tests  as  the  “probability 
of  hit,”  and  also  provides  more  information  from  the  resulting  distribution  of  measurements  (how  close  or  far  away)  than 
a  simple  hit/miss  answer.  In  many  instances,  the  probabilities  now  often  used  to  state  requirements  can  be  subsequently 
estimated  using  test  data  collected  to  evaluate  continuous  response  metrics.  Thus,  wherever  possible,  I  am  requiring  test 
plans  that  measure  continuous  performance  variables  as  the  basis  for  evaluating  thresholds  for  requirements  that  have  been 
written  in  terms  of  probabilities. 

Evaluation  Across  the  Operational  Envelope 

Another  disconnect  among  the  requirements,  test,  and  operational 
communities  is  that  often  requirements  are  narrowly-focused 
and  do  not  cover  the  operational  envelope;  a  notional  depiction 
is  shown  in  Figure  1.  To  be  adequate,  the  operational  evaluation 
must  report  performance  of  the  system  across  the  operational 
envelope,  not  just  at  single  conditions  specified  in  the  capabilities 
documents.  There  is  a  common  concern  that  failing  to  specify  a 
certain,  limited  set  of  conditions  within  requirements  could  lead 
to  an  unwieldy  test.  This  is  a  key  reason  DOT&E  is  using  Design 
of  Experiments  (DOE)  to  plan  testing  that  efficiently  spans  the 
operational  envelope.  Requirements  would  be  much  more  useful 
and  meaningful  if  they  identify  multiple  conditions  in  which  the 
system  is  likely  to  be  operated. 

1  will  continue  to  advocate  for  and  require  the  use  of  DOE  to  plan 
and  execute  tests  that  span  the  operational  envelope.  One  of  the 
key  tenets  of  a  well-designed  experiment  is  that  all  stakeholders 
must  be  engaged  in  the  detennination  of  the  goals,  metrics, 
operational  envelope,  and  test  risks.  The  requirements  community 
is  a  key  stakeholder  that  can  provide  valuable  input  regarding  what 
the  key  factors  (or  conditions)  are  that  will  most  influence  mission 
performance  and  thus  should  be  considered  in  operational  test. 

In  summary,  through  early  and  continuous  engagement  between  the  testing  and  requirements  communities,  we  can  craft 
requirements  that  are  technically  feasible,  mission-oriented,  realistic,  testable,  and  responsive  to  the  limitations  and 
opportunities  revealed  during  system  development. 


FIGURE  1 .  NOTIONAL  TWO-DIMENSIONAL  DIAGRAM  OF  A 
WEAPON  SYSTEM'S  OPERATIONAL  ENVELOPE 
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INCREASING  STATISTICAL  RIGOR  OF  T&E 

In  support  of  all  of  my  initiatives,  1  have  advocated  for  increasing  the  statistical  rigor  employed  in  planning  and  executing 
T&E.  To  that  end,  my  office  has  recently  completed  a  roadmap  to  institutionalize  Test  Science  and  statistical  rigor  in  T&E. 
The  roadmap  was  a  collaborative  activity  among  DOT&E,  Deputy  Assistant  Secretary  of  Defense  for  Developmental  Test 
and  Evaluation  (DASD(DT&E)),  the  Service  Operational  Test  Agencies  (OTAs),  and  the  Service  T&E  Organizations. 

By  increasing  statistical  rigor  and  using  state-of-the-art  test  and  analysis  methodologies,  we  will  ensure  defensible  and 
efficient  T&E.  The  Test  Science  Roadmap  accomplishes  the  following: 

•  Assesses  the  current  state  of  analytic  capabilities  within  each  of  the  Services  and  Office  of  the  Secretary  of  Defense 
(OSD) 

•  Develops  qualification  guidelines  for  personnel  performing  test  design  and  analytic  services  for  different  kinds  of  T&E 
organizations 

•  Identifies  the  training,  education,  and  other  support  that  Services  and  agencies  will  need  to  attain  the  required  test  design 
and  analytic  capabilities 

•  Develops  case  studies  of  the  implementation  of  scientific  test  design  across  test  programs 

•  Provides  guidance  for  the  documentation  of  test  design  and  statistical  rigor  in  TEMPs,  test  plans,  and  reports 

•  Forms  a  permanent  Advisory  Board  to  continually  identify  and  advocate  for  the  use  of  methods  to  incorporate  statistical 
rigor  in  all  test  planning  and  execution 

We  have  made  significant  progress  in  the  past  two  years  across  all  areas  of  the  roadmap,  as  discussed  below. 

Education  &  Training 

DASD(DT&E)  is  leading  the  way  in  improving  the  educational  materials  needed  by  our  T&E  community,  and  1  strongly 
support  them  in  this  initiative.  In  the  past  year,  we  have  added  courses  and  content  on  statistical  methods  for  T&E  to 
courses  offered  by  the  Defense  Acquisition  University.  We  have  also  made  training  widely  available  across  DOT&E, 
DASD(DT&E),  and  all  of  the  Services. 

Case  Studies  &  Best  Practices 

Case  studies  are  an  essential  educational  tool  illustrating  the  application  of  statistical  methods,  including  DOE  to  T&E. 
Over  the  past  couple  of  years,  my  office  has  developed  and  published  many  case  studies  demonstrating  the  usefulness  of 
applying  DOE  and  statistical  methods  to  T&E.  Additionally,  in  the  roadmap  meetings,  each  of  the  Services  shared  case 
studies  highlighting  the  application  of  DOE  to  solve  their  Service-specific  problems.  DOT&E  has  compiled  these  case 
studies  as  a  resource  for  the  T&E  community  (https://extranet.dote.osd.mil).  They  highlight  challenges,  areas  for  further 
research,  and  best  practices. 

Guidance  &  Policy 

Policy  that  supports  the  use  of  scientific  test  techniques  is  essential  to  ensuring  a  continued  commitment  to  Test  Science 
in  years  to  come.  Both  DASD(DT&E)  and  DOT&E  have  supported  including  more  detailed  language  in  DoDI  5000.02 
on  increasing  statistical  rigor  of  T&E.  DOT&E  also  published  a  TEMP  guidebook  highlighting  the  important  content  for 
TEMPs  and  test  plans.  This  guidance  is  available  on  the  DOT&E  public  website  (www.dote.osd.mil).  DASD(DT&E)  has 
also  taken  the  lead  on  incorporating  Test  Science  topics  into  other  guidance  documents  including  the  T&E  Management 
Guide  and  the  Guide  on  Incorporating  T&E  into  DoD  Acquisition  Contracts.  All  of  these  resources  provide  clear  and 
consistent  guidance  to  the  T&E  community  on  the  importance  of  statistics  in  T&E.  DOT&E  insists  that  TEMPs  and  test 
plans  submitted  for  approval  include  substantive  documentation  of  the  application  of  DOE  to  test  planning,  execution,  and 
evaluation. 

Advisory  Board 

Two  different  advisory  groups  have  been  formed  in  the  past  two  years.  The  first  is  the  Science  of  Test  Research 
Consortium,  funded  by  DOT&E  and  the  Director  of  Test  Resource  Management  Center;  this  academic  consortium  provides 
technical  advice  to  the  DoD  on  Test  Science  issues.  The  second  is  the  Scientific  Test  and  Analysis  Techniques  (STAT) 
Center  of  Excellence  (COE).  The  STAT  COE  funded  by  DASD(DT&E)  is  charged  with  assisting  program  managers  of 
major  acquisition  programs.  Together,  these  two  groups  are  working  to  operationalize  Test  Science  in  active  programs. 

Future  Efforts  to  Institutionalize  Statistical  Rigor 

Notwithstanding  the  significant  progress  that  has  been  made  in  the  past  two  years,  there  is  still  work  to  be  done  to  utilize 
the  full  toolset  the  scientific  community  has  available  to  support  T&E.  I  have  seen  the  Service  OTAs  modify  their  test 
design  and  planning  techniques  to  incorporate  DOE  and  take  advantage  of  the  efficiencies  afforded  by  the  use  of  its 
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methods.  Further,  I  have  observed  an  improvement  to  the  quality  of  the  TEMPs  and  test  plans  that  are  based  on  these 
methods.  However,  there  are  two  areas  requiring  improvement  as  the  Department’s  institutionalizes  statistical  rigor  in 
testing: 

•  Execution  of  testing  in  accordance  with  the  planned  test  design 

•  Analysis  of  test  data  using  the  advanced  statistical  methods  commensurate  with  test  designs  developed  using  DOE 

For  the  former,  I  have  seen  some  cases  where  a  test  is  well-designed,  but  the  desired  conditions  of  the  test  in  the  field 
are  not  the  same  as  required  by  the  original  plan.  This  has  the  effect  of  limiting  the  conclusions  that  can  be  made  from 
the  subsequent  data  or,  at  worst,  wasting  time  and  resources.  Since  most  of  our  tests  are  focused  on  characterizing  the 
performance  of  the  system  across  the  actual  conditions  in  which  the  operators  will  employ  the  system,  it  is  crucial  that 
the  planned  conditions  are  achieved  during  the  test. 

For  the  second  area,  I  have  not  yet  observed  all  of  the  OTAs  employing  the  data  analysis  methods  that  would  reap 
the  benefits  of  the  efficiencies  afforded  by  DOE.  In  other  words,  although  the  OTAs  use  statistical  rigor  in  their  test 
planning,  they  are  not  always  following  up  with  the  same  rigor  in  their  analysis  of  test  data.  The  simplest  case  of  this  is 
where  a  test  is  designed  to  cover  all  or  many  of  the  important  operational  conditions,  and  is  optimized  to  be  extremely 
efficient  in  the  number  of  test  iterations  in  each  condition,  but  the  data  analysis  is  limited  to  reporting  a  single  average 
(mean)  of  the  performance  across  all  the  test  conditions.  This  result  throws  away  all  of  the  careful  test  design  efficiencies 
afforded  by  the  use  of  DOE.  A  more  statistically  rigorous  analysis  would  enable  all  the  available  information  to  be 
extracted  from  the  data,  which  is  critical  to  evaluating  the  performance  of  systems  across  their  full  range  of  operational 
use.  The  more  advanced  statistical  analysis  also  enables  statements  of  system  performance  to  be  made  with  higher 
confidence  in  many  cases,  so  that  acquisition  decisions  can  be  based  on  certain  knowledge  rather  than  supposition. 

I  will  work  with  the  Service  OTAs  during  the  next  year  to  rectify  these  remaining  shortfalls  in  the  application  of  DOE  to 
test  execution  and  analysis. 


RELIABILITY  ANALYSIS,  PLANNING,  TRACKING,  AND  REPORTING 

Improving  system  reliability  has  been  a  DOT&E  initiative  since  2006;  the  Department  has  also  recognized  the  significant 
adverse  long-term  life  cycle  cost  impacts  and  reduced  operational  capability  resulting  from  systems  being  unreliable. 
DOT&E  initiatives  have  emphasized  the  need  for  reliability  growth  planning  and  assessment,  establishment  of  reliability 
maturity  goals  and  entrance  criteria  for  each  phase  of  testing  and  documenting  the  reliability  test  and  evaluation 
strategy  (TES)  in  the  TEMP.  Accordingly,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD(AT&L))  in  2011  released  a  Directive  Type  Memorandum  (DTM  11-03)  on  Reliability,  Analysis,  Planning, 
Tracking,  and  Reporting;  this  DTM  was  continued  into  2012  and  will  be  incorporated  into  the  updated  DoDI  5000.02 
“Operation  of  the  Defense  Acquisition  System.” 

I  am  tracking  the  impact  of  the  new 
directive  on  system  reliability.  The 
Office  of  the  Deputy  Assistant  Secretary 
of  Defense  for  Systems  Engineering 
(DASD(SE))  is  developing  an 
implementation  guide,  which  is  in  final 
staffing  and  should  be  available  in  early 
2013.  DOT&E  has  been  an  ardent 
advocate  for  the  reliability  concepts 
contained  in  the  directive,  and  has 
institutionalized  them  in  our  priorities 
and  policies.  Figure  2  plots  the  outcomes 
of  initial  operational  tests  reported  to 
Congress  for  systems  tested  between  fiscal 
years  1997  to  2012.  A  total  of  118  reports 
were  included;  each  report  includes  an 
evaluation  of  operational  effectiveness, 
operational  suitability,  and  reliability. 
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FIGURE  2.  CURRENT  TRENDS  IN  RELIABILITY 
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While  evaluations  of  operational  effectiveness  and  suitability  are  based  on  many  factors,  the  evaluations  displayed  in 
this  chart  are  based  solely  on  whether  the  system  met  its  required  reliability  threshold.  As  shown  in  Figure  2,  reliability 
continues  to  lag;  only  7/13  systems  (54  percent)  evaluated  in  2012  met  their  reliability  thresholds  and  overall  between 
1997  and  2012  only  67/118  systems  (57  percent)  were  reliable. 

To  further  understand  the  reliability  trends  in  Figure  2, 1  surveyed  52  programs  for  which  I  approved  TEMPs  or  TESs  in 
FY11  following  up  on  the  survey  1  did  in  FY10  for  all  oversight  programs.  The  TEMPs  approved  in  FY11  continue  the 
positive  trends  I  am  seeing  for  all  TEMPs  approved  after  June  2008  (when  the  Department  began  initiatives  to  improve 
reliability).  These  trends  include  programs: 

•  Having  an  approved  System  Engineering  Plan 

•  Incorporating  reliability  as  an  element  of  the  test  strategy 

•  Having  a  reliability  growth  strategy  and  documenting  it  in  the  TEMP 

•  Incorporating  reliability  and  availability  requirements 

Unfortunately,  the  programs  reviewed  in  FY 1 1  did  not  show  improvement  in  establishing  reliability-based  milestone 
or  operational  test  entrance  and  exit  criteria.  However,  I  believe  the  recent  emphasis  on  reliability  has  had  some 
demonstrable  positive  impacts.  Having  reliability  growth  curves  alone  did  not  correlate  with  attainment  of  reliability 
requirements,  but  programs  with  comprehensive  reliability  plans  were  more  likely  to  meet  their  reliability  requirements. 
A  larger  fraction  of  programs  that  establish  growth  curves  with  intermediate  goals;  anchor  milestone  or  entrance/exit 
criteria  to  reliability  performance;  use  metrics  to  ensure  reliability  growth  is  on  track;  predict  changes  caused  by  the 
implementation  of  corrective  actions;  and  calculate  reliability  growth  potential  met  their  operational  test  reliability 
entrance  and  exit  criteria  compared  to  programs  that  do  not  follow  these  practices. 

Examining  the  TEMP  survey  trends  by 
Service  shows  that  higher  percentages  of 
Army  and  Air  Force  programs:  have  added 
a  reliability  growth  strategy  since  June 
2008;  have  reliability  growth  curves;  and  are 
calculating  the  reliability  growth  potential. 

Army  and  Navy  programs  show  increasing 
improvement  in  ensuring  there  is  time  in  the 
schedule  to  implement  and  verify  corrective 
actions  and  document  the  reliability  test 
strategy.  Army  programs  are  most  likely  to: 
use  reliability  growth  curves  and  intermediate 
reliability  goals;  put  systems  into  the  hands 
of  representative  users  before  Milestone  C; 
and  document  reliability  changes  caused  by 
implementation  of  corrective  actions.  Figure 
3  shows  the  fraction  of  systems  meeting 
reliability  thresholds  at  IOT&E  for  programs 
on  DOT&E  oversight  between  1997  and 
2012  (the  same  programs  depicted  in  Figure  2  now  broken  out  by  Service). 

I  am  not  yet  seeing  more  systems  actually  meet  their  reliability  requirements  than  in  past  years,  but  I  believe  the  recent 
emphasis  on  reliability  planning  has  had  some  demonstrable  positive  impacts.  While  the  majority  of  programs  now 
have  and  are  documenting  their  reliability  growth  strategy  in  the  TEMP,  they  are  not  fully  incorporating  the  design 
for  reliability  tenets  described  in  the  ANSI/GEIA-STD-0009  Reliability  Program  Standard  for  Systems  Design, 
Development,  and  Manufacturing.  In  particular,  programs  are  failing  to  “get  on”  their  planned  reliability  growth  curve 
at  the  beginning.  I  have  seen  evidence  that  programs  with  a  procedure  for  calculating  reliability  growth  potential  (a 
calculation  that  places  emphases  on  initial  reliability,  which  in  turn  requires  that  the  system  be  designed  for  reliability) 
have  a  much  greater  likelihood  of  meeting  reliability  based  entry  criteria  for  operational  test  phases. 
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FIGURE  3.  FRACTION  OF  PROGRAMS  MEETING  RELIABILITY 
THRESHOLDS  AT  IOT&E,  BY  SERVICE  (FY97-FY12) 
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Figure  4  shows  the  distribution  of  root  failure  causes 
for  the  5 1  programs  that  did  not  meet  their  reliability 
thresholds  between  1997  and  2012.  The  root  causes 
include:  1)  inadequate  systems  management  (failures 
traceable  to  incorrect  interpretation  or  implementation 
of  requirements,  processes,  or  procedures); 
implementation  of  “bad”  requirements  (missing, 
inadequate,  ambiguous,  or  conflicting);  or  failure  to 
provide  the  resources  required  to  design  and  build  a 
robust  system;  2)  inadequate  design  margins  (failures 
resulting  from  lack  of  design  robustness  to  the  stresses 
and  loads  in  usage  environment);  3)  inadequate 
software  (failures  of  a  system  to  perform  its  intended 
function  due  to  software  issues);  4)  induced  failures 
(failures  resulting  from  externally  applied  stresses  such 
as  operator  or  maintainer  interfaces);  5)  part  quality 
(random  failures);  and  6)  manufacturing  anomalies. 

Inadequate  design  margins  and  system  management 
combine  to  account  for  76  percent  of  the  root 
causes  for  reliability  failures  in  these  data.  Clearly, 
inadequate  attention  to  reliability  during  engineering  design,  and  inadequate  management  focus  on  best  practices 
for  reliability  design  and  growth  testing  have  been  and  continue  to  remain  a  concern  -  improvements  in  these  areas, 
particularly  using  a  Design  For  Reliability  strategy,  would  help  programs  get  on  their  planned  reliability  growth  curve 
and  have  a  greater  likelihood  of  meeting  their  ultimate  reliability  goals.  Additionally,  software  reliability  design  and 
growth  testing  are  of  concern.  The  12  percent  of  systems  that  failed  due  to  software  root  causes  in  Figure  4  are  mostly 
software  intensive  systems  like  APG-79  Active  Electronically  Scanned  Array  (AESA)  Radar  (software  immaturity 
causes  excessive  and  inexplicable  radar  hang-ups;  the  built-in  test  function  is  not  automated  to  isolate  software  failures); 
F-15  Mission  Planning  System  (suitability  is  poor  due  to  software  instability,  high  frequency  of  system  crashes);  and 
Large  Aircraft  Infrared  Countermeasures  (LA1RCM)  Phase  II  (software  design  bugs  caused  19  critical  failures;  bugs 
were  traced  to  software  coding  errors). 
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FIGURE  4.  ROOT  FAILURE  CAUSES  FOR  THE  51  PROGRAMS  NOT 
MEETING  RELIABILITY  THRESHOLDS  BETWEEN  FY97  AND  FY12 


SOFTWARE  TESTING 

I  continue  to  see  software  issues  in  programs  of  all  types.  Most  commonly,  programs  do  not  create  adequate  ability  to 
track  software  reliability  and  test  software  patches.  Software  requirements  are  poorly  stated  and  in  some  cases  wrongly 
tested.  There  are  also  unique  needs  for  the  special  class  of  programs,  business  systems,  which  are  being  developed  by 
the  Services  to  meet  the  2014  and  2017  Congressional  deadlines  for  auditability. 

Software  Reliability 

Software  reliability  is  broadly  similar  to  reliability  for  any  weapon  system  with  subtle  distinctions  in  failure  definitions, 
defect  tracldng,  and  the  speed  of  the  test-fix-test  cycle.  The  overall  effect  of  these  distinctions  has  led  me  to  conclude 
that  new  policy  is  needed  that  will  the  mandate  the  use  of  some  software  test  automation  for  most  programs  that  utilize 
software. 

Failure  Definition  and  Defect  Tracking:  Software  is  nearly  always  multi-functional.  Software  use  is  not  well 
represented  by  failure-per-hour  metrics.  Except  in  cases  where  the  same  operation  is  performed  repeatedly  (for 
example  spacecraft  during  planetary  cruise),  programs  shoidd  simply  track  counts  of  defects.  Defects  should  always  be 
categorized  by  severity  in  accordance  with  Institute  for  Electrical  and  Electronics  Engineers  standards.  Programs  should 
track  open  and  closure  rates  of  the  defects  in  each  category.  For  multi-functional  systems,  it  is  helpful  to  track  defects 
against  distinct  capabilities  as  well. 

Test-Fix-Test:  The  test-fix-test  cycle  for  software  is  faster  and  less  visible  than  for  other  systems  types.  For  many 
software  issues,  there  is  no  meaningful  distinction  between  maintenance  and  follow-on  development.  Given  the  speed 
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of  software  development,  the  inability  to  oversee  software  in  detail,  and  the  fact  that  one  must  develop  code  to  fix  code, 
the  line  between  fixing  defects  and  adding  features  is  nearly  always  blurred.  Given  the  pace  at  which  new  security 
patches  and  product  updates  and  changes  in  the  computing  environment  occur,  there  is  also  essentially  no  such  thing  as 
a  stable  software  system.  For  all  of  these  reasons,  I  have  concluded  that  operational  testing  of  software  must  include  a 
demonstration  of  the  program’s  ability  to  perform  robust  and  repeatable  testing  in  support  of  software  maintenance. 

In  support  of  robust  and  repeatable  within-program  testing,  I  have  begun  enforcing  the  following  test  automation 
policies,  which  will  be  contained  in  the  next  version  of  the  DoDl  5000.02: 

•  At  Milestone  A,  program  managers  shall  identify  an  approach  to  software  test  automation,  including  when  key 
test  automation  software  components  or  services  will  he  acquired  and  how  those  decisions  will  be  made.  The  test 
automation  approach  shall  be  updated  in  the  Milestone  B  and  Milestone  C  TEMPs  as  appropriate. 

•  Program  managers  shall  demonstrate  system  sustainment  maturity  at  IOT&E.  Sustainment  maturity’  shall  include 
routine  T&E  to  support  routine  technology  upgrades.  For  Information  Systems,  Defense  Business  Systems,  and 
software  components  of  Weapons  Systems,  program  managers  shall  demonstrate  mature  test  automation  to  include  an 
end-to-end  trace  of  test  information  from  requirements  to  test  scripts  to  defect  tracing 

This  year,  I  recommended  the  following  programs  demonstrate  this  test-fix-test  cycle:  Next  Generation  Enterprise 
Network  (NGEN),  Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN),  Defense  Enterprise  Accounting 
and  Management  System  (DEAMS),  EProcurement,  and  Global  Combat  Support  System  -  Army  (GCSS-Army). 
Because  the  development  of  automation  tools  can  be  time  consuming  given  the  complexity  of  many  of  these  programs, 

I  anticipate  that  most  programs  will  take  several  years  to  create  an  automated  test-fix-test  approach  to  satisfy  these 
recommendations.  Currently,  very  few  acquisition  programs  have  mature  test  automation  solutions  for  regression 
testing  that  can  be  demonstrated  at  IOT&E  and  even  fewer  can  create  the  environments  and  conditions  to  validate  their 
regression  testing  processes.  Without  substantial  help  from  a  central  resource,  it  is  likely  that  most  programs  will  have 
this  deficiency  assessed  during  IOT&E. 

The  need  for  test  automation  will  create  demand  for  the  corresponding  expertise  in  program  offices.  Program  managers 
need  a  resource  in  the  form  of  a  center  of  excellence  to  help  meet  that  demand,  and  DOT&E  is  taldng  the  initial  steps  to 
establish  such  a  center.  The  center  of  excellence  would  work  with  vendors  and  government  providers  to  promote  the  use 
of  various  test  automation  solutions  under  the  construct  of  “Test  as  a  Service  (TaaS).”  A  center  of  excellence  will: 

•  Centralize  knowledge  of  the  various  automation  approaches 

•  Assist  programs  in  applying  test  automation 

•  Create  "in-house"  test  automation  expertise 

A  center  of  excellence  TaaS  capability  may  lessen  the  tendency  of  program  offices  and  vendors  to  use  a  “stove-piped” 
approach  to  test  automation,  may  reduce  duplicative  resources  (technological  and  human),  should  increase  programs’  use 
of  existing  capabilities,  and  should  improve  the  consistency  and  adequacy  in  the  types  of  testing  accomplished. 

Testers  do  not  have  questions  about  system  maturity  that  are  distinct  from  the  questions  the  systems  managers  should 
have.  System  managers  should  always  know  how  well  the  system  is  functioning.  If  testers  have  reasonable  questions 
about  system  performance  that  the  system  managers  cannot  answer  with  the  data  they  are  already  gathering,  then 
the  system  management  probably  is  not  as  mature  as  it  should  be.  Examples  of  performance  parameters  that  should 
be  routinely  and  continuously  reported  to  the  system  management  include  defect  tracking,  helpdesk  use,  system 
productivity/utilization,  schedule  of  upcoming  changes  (commercial  releases,  changes  in  interfacing  system,  etc.),  staff 
turnover  rate,  and  training  and  documentation  adequacy. 

Software  Requirements 

Software  requirements  typically  involve  KPPs  for  system  response  time,  data  loss  and  restoration,  and  data  transmission 
accuracy.  DOT&E  has  seen  many  examples  of  metrics  that  incorrectly  capture  this  information.  For  example,  a  KPP 
might  specify  95  percent  accuracy  for  information  retrieval;  but  if  a  random  5  percent  of  your  data  is  garbled  every  time 
you  use  the  system,  the  utility  of  that  system  is  very  much  in  question.  Some  programs  include  requirements  for  data 
loss  in  event  of  an  outage  or  other  emergency  that  requires  a  system  restore  from  backups,  and  these  are  almost  always 
expressed  as  percentages.  The  data  loss  requirements  should  be  expressed  in  time,  not  percentages.  In  every  case,  the 
system  is  expected  to  lose  100  percent  of  the  data  that  has  been  entered  following  the  most  recent  backup  interval  before 
the  outage.  I  have  seen  that  testers  are  dutifully  reporting  the  amount  of  data  loss,  but  that  is  not  meaningful.  Rather, 
testers  should  always  perform  a  demonstration  test  that  verifies  the  ability  of  the  system  to  backup  and  restore  data  on  a 
schedule  consistent  with  the  operational  need  for  the  system  to  be  available  for  use. 
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Finally,  some  programs  have  percentage  KPPs  for  data  accuracy.  These  KPPs  reference  a  variety  of  technical 
circumstances:  transmission  across  interfaces,  retrieval  from  databases,  account  balances,  and  so  on.  These  are  often 
treated  as  global  metrics  but  they  should  be  treated  as  percentages  that  apply  to  some  relevant  set  of  channels.  For 
example,  in  general,  once  an  interface  is  correct  it  is  always  correct.  It  is  much  less  important  to  know  that  95  percent 
of  the  data  transmitted  across  all  interfaces  is  correct  than  it  is  to  know  which  5  percent  of  the  interfaces  are  transmitting 
incorrectly.  The  metric  should  not  simply  be  the  global  number  of  errors  per  the  number  of  transmissions.  The  mission 
need  is  for  the  data  elements  with  errors  to  be  limited.  Therefore,  the  metrics  should  be  looking  at  counts  of  element 
types  containing  errors.  Global  metrics  also  contribute  less  to  finding  and  fixing  problems  than  woidd  differentiated 
metrics. 

Vulnerability  of  Business  Systems 

The  HOUSE  ARMED  SERVICES  COMMITTEE  PANEL  ON  DEFENSE  FINANCIAL  MANAGEMENT  AND 
AUDITABILITY  REFORM  FINDINGS  AND  RECOMMENDATIONS  (January  24,  2012),  Recommendation  4.9 
directed  DOT&E  and  others  to  identify  and  address  shortfalls  in  workforce  levels  and  corresponding  skill  sets  for 
Enterprise  Resource  Programs  (ERPs).  A  clear  shortfall  in  the  testing  of  these  systems  is  in  identification  of  financial 
vulnerabilities.  I  have  accordingly  begun  directing  that  the  financial  vulnerabilities  of  ERPs  be  probed  in  a  manner 
analogous  to  Information  Assurance,  and  anticipate  that  such  testing  will  draw,  at  least  initially,  on  the  existing 
commercial  services  that  provide  such  testing.  The  programs  to  which  this  applies  are: 

•  Air  Force  Integrated  Personnel  and  Pay  System  (AF-IPPS) 

•  Defense  Agency  Initiative  (DAI) 

•  Defense  Enterprise  Accounting  and  Management  System  -  Increment  1  (DEAMS  -  Increment  1) 

•  Defense  Enterprise  Accounting  and  Management  System  -  Air  Force  (DEAMS  -  AF) 

•  EProcurement 

•  Future  Pay  and  Personnel  Management  Solution  (FPPS  -  Navy)  Pre-MAIS 

•  General  Fund  Enterprise  Business  System  (GFEBS) 

•  Global  Combat  Support  System  -  Army  (GCSS  -  Army) 

•  Integrated  Personnel  and  Pay  System  -  Army  (Army  1PPS) 

•  Navy  Enterprise  Resource  Planning  (ERP) 

In  support  of  this  initiative,  the  DCMO  has  initiated  a  study  of  commercial  providers  of  financial  Red  Team  test  services. 
In  general,  commercial  vendors  of  these  services  focus  on  protect  and  detect  capabilities  (both  system  and  people).  They 
work  with  their  clients  to  identify  likely  targets  for  fraud  or  theft  within  the  system;  they  may  attempt  (within  established 
rules  of  engagement)  to  circumvent  controls  and  processes;  and  they  assess  the  audit  processes  that  are  in  place  to  catch 
fraud  or  theft.  In  addition,  together  with  the  Deputy  Chief  Management  Officer,  DASD(DT&E),  and  DASD(SE),  we 
will  ensure  that  developmental  and  operational  testing  helps  fulfill  the  Federal  Information  System  Controls  Audit 
Manual  requirements. 


OTHER  AREAS  OF  INTEREST 

Electronic  Warfare  Test  Infrastructure 

In  February  2012, 1  identified  shortfalls  in  electronic  warfare  test  resources  that  prevent  adequate  developmental  and 
operational  testing  of  many  systems,  including,  but  not  limited  to,  the  Joint  Strike  Fighter.  I  am  working  to  address 
these  shortfalls  in  government  anechoic  chambers,  open-air  ranges,  and  the  Joint  Strike  Fighter  electronic  warfare 
programming  laboratory.  My  staff  participated  in  a  “tiger  team”  assigned  by  the  USD(AT&L)  to  review  the  issue,  which 
concurred  with  my  conclusions  and  recommended  additional  enhancements. 

Cyber  Testing 

Implementation  of  the  February  2011  Chairman  of  the  Joint  Chiefs  Execute  Order  (EXORD),  which  directed  increased 
cyber-adversary  realism  for  training  events,  has  been  modest.  During  FY12,  most  of  the  exercise  assessments  and  tests 
involved  operations  largely  against  low-  and  mid-level  cyber  threats  and  on  networks  that  were  only  moderately  stressed 
in  terms  of  loading  or  network  degradation.  In  the  cases  where  the  adversary  team  portrayed  higher-level  threats, 
exercise  training  audiences  frequently  misinterpreted  these  portrayals  as  maintenance  issues,  poor  system  performance, 
or  anomalies.  This  indicates  that  the  Department  has  not  yet  developed  sufficiently  advanced  cyber  defensive  tactics  to 
counter  advanced  adversary  tactics  and  to  consistently  operate  in  degraded  cyber  environments.  Following  publication 
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of  the  FY11  Annual  Report,  I  provided  a  separate  and  classified  amplification  of  findings,  which  resulted  in  a  series 
of  meetings  with  the  Deputy  Secretary  of  Defense  on  the  topic  of  how  these  operational  and  training  shortfalls  might 
be  resolved.  A  number  of  actions  resulting  from  these  discussions  are  in  progress,  including  the  consolidation  and 
enhancement  of  training  support  capabilities,  additional  guidance  on  meeting  the  intent  and  requirements  of  the  EXORD, 
and  improving  the  way  the  Department  ensures  that  critical  shortfalls  are  resolved.  Additionally,  the  lessons  garnered 
from  operational  network  assessments  are  being  applied  to  the  acquisition  and  testing  of  information  systems  to  ensure 
that  subsequent  systems  procurement  does  not  contain  cyber  shortfalls  already  discovered  and  documented  by  the 
Department.  I  also  remain  closely  engaged  with  U.S.  Cyber  Command  and  other  key  stakeholders  to  ensure  priority  is 
given  to  the  necessary  investments  supporting  improved  Red  Team  availability,  capability,  and  accessibility. 

Testing  Protocols  for  Personal  Protective  Equipment 

I  continue  to  exercise  oversight  over  the  testing  of  personal  protective  equipment.  The  National  Academy  of  Sciences’ 
Committee  to  Review  the  Testing  of  Body  Armor  published  its  final  report  in  May  of  2012  and  I  and  the  Services  are 
pursuing  the  report’s  recommendations.  Congressional  interest  in  the  testing  of  the  Army’s  Advanced  Combat  Helmet 
(ACH)  resulted  in  the  Department’s  Inspector  General  initiating  a  technical  assessment  of  the  ACH.  In  response  to  this 
Congressional  interest  in  the  ACH,  we  have  also  asked  the  NRC  to  conduct  an  independent  review  of  the  helmet  testing 
protocols.  This  is  a  direct  follow-up  to  the  NRC’s  independent  review  of  hard  body  armor  testing,  which  included  a 
review  of  test  protocols.  One  of  the  objectives  of  the  review  is  to  examine  the  rigor  of  statistical  metrics.  My  staff 
will  leverage  the  knowledge  of  some  of  the  nation’s  leading  statisticians  to  improve  and  advance  the  use  of  statistical 
techniques  in  test.  I  will  also  conduct  a  comprehensive  technical  assessment  of  the  ACH  to  characterize  its  ballistic 
performance  more  comprehensively  than  is  possible  with  existing  data.  The  results  of  these  assessments  will  provide  the 
basis  for  any  changes  to  the  current  helmet  test  protocols  that  might  be  appropriate. 

Warrior  Injury  Assessment  Manikin  (WlAMan) 

I  am  sponsoring  a  five-year  research  and  development  program  to  increase  the  Department’s  understanding  of  the 
cause  and  nature  of  injuries  incurred  in  underbody  blast  combat  events  and  to  develop  appropriate  instrumentation 
to  assess  such  injuries  in  testing.  This  program,  known  as  the  Warrior  Injury  Assessment  Manikin,  utilizes  expertise 
across  multiple  commands  and  disciplines  within  the  Army  to  generate  a  medical  research  plan  from  which  data  will, 
at  pre-determined  times,  be  transitioned  to  the  materiel  and  T&E  communities.  These  data  will  feed  the  design  of  a 
biofidelic  prototype  Anthropomorphic  Test  Device  designed  to  capture  occupant  loading  from  the  vertical  direction, 
reflecting  the  primary  load  axis  to  which  occupants  are  exposed  in  an  under-vehicle  blast  event. 

Environment  and  Renewable  Energy  Effects  on  Test  Ranges 

The  Department’s  ranges  are  experiencing  encroachment  from  infrastructure  associated  with  the  electrical  energy 
production  and  transmission  industry.  This  encroachment  can  affect  test  operations  as  well  as  systems  under  test  through 
a  variety  of  means.  These  include  physical  obstructions,  electromagnetic  interference,  and  thermal  effects.  The  sources 
of  such  encroachment  include  wind  turbines,  solar  power  towers,  photovoltaic  panels,  and  high  voltage  bulk  power 
transmission  lines.  I  will  continue  to  cooperate  with  the  Department’s  Siting  Clearing  House  and  the  Services  to  identify 
potential  encroachment  of  our  ranges  resulting  from  renewable  energy  infrastructure  and  work  to  mitigate  the  impact  of 
such  encroachment. 


CONCLUSION 

Since  my  first  report  to  you  in  2009,  we  have  made  significant  progress  increasing  the  scientific  and  statistical  rigor  of 
OT&E;  we  have  engaged  early  with  the  requirements  community  to  develop  realistic,  feasible,  and  testable  requirements; 
we  have  focused  attention  on  reliability  management,  design,  and  growth  testing;  and  we  continue  to  support  rapid 
fielding  through  flexible  and  early  operational  test  events.  I  submit  this  report,  as  required  by  law,  summarizing  the 
operational  and  live  fire  T&E  activities  of  the  Department  of  Defense  during  FY12. 

J.  /¥.  — - 

J.  Michael  Gilmore 
Director 
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DOT&E  ACTIVITY  AND  OVERSIGHT 


Activity  Summary 


DOT&E  activity  for  FY12  involved  oversight  of  327  programs, 
including  41  Major  Automated  Information  Systems  (MAIS). 
Oversight  activity  begins  with  the  early  acquisition  milestones, 
continues  through  approval  for  full-rate  production  and,  in  some 
instances,  during  full  production  until  deleted  from  the  DOT&E 
oversight  list. 

Our  review  of  test  planning  activities  for  FY12  included  approval 
of  43  Test  and  Evaluation  Master  Plans  (TEMPs)  and  1  Test 
and  Evaluation  Strategy,  as  well  as  73  Operational  Test  Plans 
and  1  Live  Fire  Test  and  Evaluation  (LFT&E)  Management 
Plan.  In  FY12,  DOT&E  prepared  for  the  Secretary  of  Defense 
and  Congress  14  Beyond  Low-Rate  Initial  Production  (BLRIP) 
reports,  2  Early  Fielding  reports,  6  Follow-on  Operational  Test 
and  Evaluation  (FOT&E)  reports,  2  LFT&E  reports,  1  MAIS 
report,  and  2  special  reports,  as  well  as  the  Ballistic  Missile 
Defense  (BMD)  Programs  FY 1 1  Annual  Report.  Additional 
FY12  DOT&E  reports  that  did  not  go  to  Congress  included 
25  Operational  Assessment  reports,  1  FOT&E  report,  1  LFT&E 
report,  6  MAIS  reports,  and  3  special  reports. 


DOT&E  also  prepared  and  submitted  numerous  reports  to  the 
Defense  Acquisition  Board  (DAB)  principals  for  consideration  in 
DAB  deliberations. 

During  FY12,  DOT&E  met  with  Service  operational  test 
agencies,  program  officials,  private  sector  organizations,  and 
academia;  monitored  test  activities;  and  provided  information  to 
the  DAB  committees  as  well  as  the  DAB  principals,  the  Secretary 
and  Deputy  Secretary  of  Defense,  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics),  the  Service  Secretaries, 
and  Congress.  Active  onsite  participation  in,  and  observation 
of,  tests  and  test-related  activities  are  a  primary  source  of 
information  for  DOT&E  evaluations.  In  addition  to  onsite 
participation  and  local  travel  within  the  National  Capital  Region, 
approximately  925  trips  supported  the  DOT&E  mission. 

Security  considerations  preclude  identifying  classified  programs 
in  this  report.  The  objective,  however,  is  to  ensure  operational 
effectiveness  and  suitability  do  not  suffer  due  to  extraordinary 
security  constraints  imposed  on  those  programs. 


TEST  AND  EVALUATION  MASTER  PLANS  /  STRATEGIES  APPROVED 


Advanced  Extremely  High  Frequency  (AEHF)  Satellite 
Communications  System  TEMP 

Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM)  Electronic 
Protection  Improvement  Program  (EPIP)TEMP 

Air  and  Space  Operations  Center  -  Weapon  System  (AOC-WS), 
Increment  10.1  TEMP 

Air  Force  Integrated  Personnel  and  Pay  System  (AF-IPPS)TEMP 
Amphibious  Assault  Ship  Replacement  (LHA  (R))  TEMP 
AN/ALR-69A  Radar  Warning  Receiver  Program  TEMP 
Apache  Block  III  (AB3)  TEMP 
Army  Integrated  Air  and  Missile  Defense  TEMP 

Automatic  Radar  Periscope  Detection  and  Discrimination  (ARPDD) 
Upgrade  TEMP 

B-61  Life  Extension  Program  Tailkit  Assembly  Test  and  Evaluation 
Strategy 

Battle  Control  System  -  Fixed  (BCS-F),  Increment  3,  Release  3.2  (R3.2) 
TEMP 

Cooperative  Engagement  Capability  (CEC)  TEMP  Revision  5 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS) 
TEMP 

Distributed  Common  Ground  System  -  Army  (DCGS-A)  Increment  1 
TEMP 

Distributed  Common  Ground  System  -  Marine  Corps  (DCGS-MC) 
Milestone  BTEMP 


Enhanced  AN/TPQ-36  (EQ-36)  Milestone  C  Update  TEMP 

EProcurement  (EProc)  TEMP 

EProcurement  Addendum  to  Milestone  C TEMP 

F/A-1 8E/F  and  EA-1 8G  Flight  Plan  TEMP  No.  1 787 

Global  Broadcast  Service  (GBS)TEMP 

Global  Positioning  System  (GPS)  Enterprise  TEMP  (ETEMP) 

Ground/AirTask  Oriented  Radar  (G/ATOR)  TEMP 

Guided  Multiple  Launch  Rocket  System  -  Alternate  Warhead 
(GMLRS-W)  Milestone  BTEMP 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  Suite 
Block  4TEMP 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  Suite 
Block  4TEMP  No.  1490  Annex  C  Change 

Integrated  Submarine  Imaging  System  (ISIS)  Program  TEMP 
Joint  High  Speed  Vessel  (JHSV)TEMP 

Joint  Lightweight  Tactical  Vehicle  (JLTV)  Increment  I  Milestone  BTEMP 
Joint  Warning  and  Reporting  Network  (JWARN)  Increment  I  TEMP 
Key  Management  Infrastructure  (KMI)  Increment  2 TEMP 
Ml  09  Family  of  Vehicles  Paladin  Integrated  Management  (PIM)TEMP 
Mark  XIIA  Mode  5  Identification  Friend  or  Foe  (IFF)  TEMP 

Mk48  Mod  7  Common  Broadband  Advanced  Sonar  System  (CBASS) 
and  the  Mk  48  Mod  6  Advanced  Common  Torpedo  with  Advanced 
Processor  Build  (APB)  Spiral  4 TEMP  No.  0371 
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MQ-1C  Increment  1  Gray  Eagle  (GE)  Unmanned  Aircraft  System  (UAS) 
TEMP 

MQ-4C  Broad  Area  Maritime  Surveillance  (BAMS)  Unmanned  Aircraft 
System  (UAS)  TEMP 

MQ-9  Reaper  Increment  1  Unmanned  Aircraft  System  TEMP 
MQ-9  Reaper  Increment  1  Unmanned  Aircraft  System  TEMP  Update  1 

RC-135  Multi-functional  Information  Distribution  System  (MIDS)  Joint 
Tactical  Radio  System  (JTRS)  Integration  TEMP 

Remote  Minehunting  System  (RMS)  TEMP  Revision  D 
Rolling  Airframe  Missile  (RAM)  Block  2 TEMP 
Ship-to-Shore  Connector  (SSC)  TEMP 


Space-Based  Infrared  System  (SBIRS)  Enterprise  TEMP  (ETEMP) 
Submarine  Electronic  Warfare  Support  (ES)  System  (AN/BLQ-10)TEMP 

Surface  Mine  Countermeasures  Unmanned  Undersea  Vehicle  (SMCM 
UUV)  TEMP  (February  201 2) 

Surface  Mine  Countermeasures  Unmanned  Undersea  Vehicle  (SMCM 
UUV)  TEMP  (August  2012) 

Teleport,  Generation  3,  Phase  2  (G3P2)TEMP  Update 

Virginia  (SSN  774)  TEMP  with  Design  of  Experiments  (DOE)  Appendix 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2  TEMP 


OPERATIONAL  TEST 


Advanced  Extremely  High  Frequency  (AEHF)  Operational  Utility 
Evaluation  (OUE)  2  Operational  Test  Agency  (OTA)  Test  Plan 

Aegis  Baseline  7.1  R  and  Cooperative  Engagement  Capability  (CEC) 
FOT&ETest  Plans 

Air  and  Space  Operations  Center  (AOC)  -  Weapon  System  (WS) 
Increment  10.1  Recurring  Event  (RE)  1 1  Force  Development  Evaluation 
(FDE)  Plan 

Air  and  Space  Operations  Center  (AOC)  -  Weapon  System  (WS) 
Increment  10.1  Overarching  OT&E  Plan 

Air  Intercept  Missile-9X  Block  II  IOT&E  Plan 

Airborne  Warning  and  Control  System  (AWACS)  Block  40/50  Upgrade 
IOT&E  Plan 

AN/ALR-69A  Radar  Warning  Receiver  (RWR)  IOT&E  Test  Plan 

Apache  Block  III  Force  Development  Test  and  Experimentation  II 
(FDT&E  II)  and  IOT&E  Operational  Test  Agency  (OTA)  Test  Plan 

Automatic  Radar  Periscope  Detection  and  Discrimination  (ARPDD) 
Upgrade  IOT&E  Plan 

B-2  EHF  SATCOM  Increment  1  IOT&E  Plan 

Battle  Control  System  -  Fixed  Release  3.2  (BCS-F  R3.2)  IOT&E  Plan 

C-1 30  Avionics  Modernization  Program  IOT&E  Test  Plan 

C-1 30J  Data  Transfer  and  Diagnostics  System  (DTADS)  FOT&E  Plan 

C-130J  Station  Keeping  Equipment  (SKE)  Software  Enhancement  (SSE) 
FOT&E-2  Plan 

C-5  Reliability  Enhancement  and  Re-Engining  Program  Operational 
Flight  Program  3.5  FDE  Test  Plan 

CNO  Project  No.  1714,  Enterprise  Test  (ET-03)  Phase  2  of  the  Air 
Warfare/Ship  Self-Defense  (AW/SSD)  Enterprise,  CNO  Project  1400, 
FOT&E  (OT-IIIF)  of  the  Ship  Self-Defense  System  (SSDS)  Mark  2  Mod  1 A 
and  CNO  Project  1471,  FOT&E  (OT-D2)  of  the  Evolved  SeaSparrow 
Missile  (ESSM)  Program 

CNO  Project  No.  1 787,  FOT&E  OT-D1  of  the  EA-1 8G  Airborne  Electronic 
Attack  Aircraft  System  Configuration  Set  H8ETest  Plan 

CNO  Project  No.  3980  1 552-OT-B2/1 583-OT-B2,  Operational 
Assessment  (OA)  (OT-B2)  of  the  MH-60S  Block  2  Airborne  Mine 
Countermeasures  System  (AMCM)  and  the  AN/AES-1  Airborne  Laser 
Mine  Detection  System  (ALMDS)Test  Plan 


PLANS APPROVED 

CNO  Project  No.  J1656,  MOT&E  (OT-D3)  of  the  Mobile  User  Objective 
System  (MUOS)Test  Plan 

Common  Aviation  Command  and  Control  System  (CAC2S), 

Increment  1,  Phase  1  Limited  User  Evaluation  Plan 

Consolidated  Afloat  Networks  and  Enterprise  Services  (CANES)  OA 
Test  Plan 

Cooperative  Engagement  Capability  (CEC)  FOT&E  (OT-IIIF)  Test  Plan 

Distributed  Common  Ground  System  -  Army  (DCGS-A)  Increment  1 , 
DCGS-A  Software  Baseline  (DSB)  1.0  IOT&E  OTA  Test  Plan  (TP) 

Distributed  Common  Ground  System  -  Navy  (DCGS-N)  FOT&E  Plan 

Dry  Cargo  and  Ammunition  Ship  (T-AKE)  Program  Change 
Transmittal  II  to  Test  Plan 

E-2D  Advanced  Hawkeye  (AHE)  IOT&E/CEC  FOT&ETest  Plan 
EProcurement  Release  1.2  IOT&E  Plan 
EProcurement  Release  1 .2  OATest  Plan 

F/A-1 8/F  System  Configuration  Set  (SCS)  H8E  Software  Qualification 
Test  (SQT)  And  Active  Electronically  Scanned  Array  (AESA)  Radar 
Upgrade  Phase  III  OT&E  Plan 

Gerald R.  Ford  Class  CVN-78  Aircraft  CarrierTest  Plan  for  OA  OT-B3 

Global  Command  and  Control  System  -  Joint  (GCCS-J)  Global 
Version  4.2.0.9  Release  OT&E  Plan 

H-1  Upgrades  Program  FOT&E  (OT-IIIB) Test  Plan 
HC/MC-130J  Recapitalization  (RECAP)  IOT&E  Plan 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  Suite 
Block 4  OATest  Plan 

Integrated  Personnel  and  Pay  System -Army  (IPPS-A)  Test  and 
Evaluation  Plan  (TEP) 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN) 

Increment  2  IOT&E  Plan 

Joint  Chemical  Agent  Detector  (JCAD)  Visit,  Board,  Search,  Seizure 
(VBSS)  OTA  Test  Plan  for  Developmental  Test/Operational  Test  (DT/OT) 

Joint  Space  Operations  Center  (JSpOC)  Mission  System  (JMS) 
Increment  1  OUE  OTA  Test  Plan 

Joint  Space  Operations  Center  (JSpOC)  Mission  System  (JMS) 
Increment  2  Early  OA  (EOA)  Plan 


2  Activity 


DOT&E  ACTIVITY  AND  OVERSIGHT 


Joint  Tactical  Radio  System  (JTRS)  Enterprise  Network  Manager  (JENM) 
for  Soldier  Radio  Waveform  (SRW)  IOT&E  Test  Plan 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  Small  Form  Fit 
(HMS)  Manpack  Radio  MOT&E  OTA  Test  Plan 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  Small  Form  Fit 
(HMS)  Rifleman  Radio  IOT&E  OTA  Test  Plan 

Joint  Warning  and  Reporting  Network  (JWARN)  Increment  1  FOT&E 
OTA  Test  Plan 

Key  Management  Infrastructure  (KMI)  Increment  2,  Spiral  1,  IOT&E 
Plan 

Littoral  Combat  Ship  (LCS)  Quick  Reaction  Assessment  (QRA)  Data 
Management  and  Analysis  Plan  (DMAP) 

LPD-1 7  Data  Management  and  Analysis  Plan  (DMAP)  for  Deficiency 
concerning  reliability  during  the  first  five  hours  of  Amphibious  Assault 
(R5) 

LPD-1 7  FOT&E  Test  Plan  for  Chemical,  Biological,  Radiological  Defense 
(CBRD)  and  Magnetic  Signature  Check  Range  Run 

Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5  Joint  Operational 
Test  Approach  (JOTA)  Test  Plan 

Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5  Joint  Operational 
Test  Approach  (JOTA)  Version  2.0 

Miniature  Air-Launched  Decoy  -  Jammer  (MALD-J)  ADM-1 60C  IOT&E 
Test  Plan 

Mk  48 Test  Plan 

Mk  48  Mod  6  Advanced  Common  Torpedo  (ACOT)  and  Mk  48  Mod  7 
Common  Broadband  Advanced  Sonar  System  (CBASS)  Torpedo 
FOT&E  Test  Plan 

Mk  54 Test  Plan  Change  1 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  IOT&E  OTA  Test  Plan 
MQ-9  Operational  Flight  Program  904.2  FDE  Plan 


MV-22B  Block  C  FOT&E  Test  Plan 

Ohio  Class  Replacement  Submarine  EOATest  Plan  (1 771 -OT-A1 ) 

P-8A  Poseidon  Multi-mission  Maritime  Aircraft  (MMA)  IOT&E  Test  Plan 
Patriot  Limited  UserTest  (LUT)  OTA  Test  Plan 

RC-135  Multi-functional  Information  Distribution  System  (MIDS)  Joint 
Tactical  Radio  System  (JTRS)  Integration  Test  Plan 

Small  Diameter  Bomb  Increment  II  (SDB  II)  Multi-Service  OA  Plan 
Space  Fence  EOA  OTA  Test  Plan 

Space-Based  Infrared  System  Effectivity  5  OUE  OTA  Test  Plan 

Standard  Missile-6  (SM-6)  OT-IIB  lOT&ETest  Plan  Annex  A  Model  and 
Simulation  Runs  for  the  Record  (RFR)  (U) 

Stryker  Double-VHull  (DVH)  Mortar  Carrier  Vehicle  (MCW)  OTA  Test 
Plan 

Stryker  Double-V  Hull  (DVH)  Addendum  to  the  LFT&E  Phase  2/3  OTA 
Test  Plan  and  the  Detailed  Test  Plan  (DTP) 

Submarine  Electronic  Warfare  Support  (ES)  System  (AN/BLQ-10) 
Integrated  Evaluation  Framework  (IEF) 

Surveillance  Towed  Array  Sensor  System  (SURTASS)  Compact  Low 
Frequency  Active  (CLFA)Test  Plan  for  IOT&E  (OT-IIG) 

Teleport  Generation  3,  Phase  2  (G3P2)  OA  OTA  Plan 

U.S.  Air  Forces  Central  (AFCENT)  Combined  Air  and  Space  Operations 
Center  (CAOC)  Information  Assurance  Assessment  Plan 

Vertical  Take-off  and  Landing  Tactical  Unmanned  Aerial  Vehicle 
(VTUAV)  System  Quick  Reaction  Assessment  (QRA)  Data  Management 
and  Analysis  Plan  (DMAP) 

Warfighter  Exercise  (WFX)  12-4  Information  Assurance  and 
Interoperability  Assessment  Plan 

Warfighter  Information  Network -Tactical  Increment  2  IOT&E  OTA  Test 
Plan 


LIVE  FIRE  TEST  AND  EVALUATION  STRATEGIES,  TEST  PLANS,  AND  MANAGEMENT  PLANS 

57  mm  Ammunition  LFT&E  Management  Plan 
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FY12  DOT&E  REPORTS  TO  CONGRESS 

Program 

Report  Type 

Date 

BLRIP  Reports 

Stryker  Nuclear,  Biological,  and  Chemical  Reconnaissance  Vehicle  (NBCRV) 

Combined  OT/LFT 

November  201 1 

Navy  Multiband  Terminal  (NMT)  with  Classified  Annex 

OT  Report 

November  201 1 

Large  Aircraft  Infrared  Countermeasure  (LAIRCM)  Phase  II  System 

OT  Report 

January  2012 

Global  Positioning  System  Selective  Availability/Anti-Spoof  Module  (GPS  SAASM) 

Multi-Service  OT  Report 

February  2012 

Terminal  High-Altitude  Area  Defense  (THAAD)  and  AN/TPY-12  Radar 

Combined  OT/LFT 

February  2012 

Spider  XM7  Network  Command  Munition  with  Confidential  Annex 

Combined  OT/LFT 

February  2012 

Mine  Resistant  Ambush  Protected  (MRAP)  Family  of  Vehicles:  Dash  with 

Independent  Suspension  System  (ISS),  MRAP  Recovery  Vehicle  (MRV),  Marine  Corps 
Cougar  Ambulance 

Combined  OT/LFT 

March  2012 

Global  Combat  Support  System  -  Army  (GCSS-Army) 

OT  Report 

June  201 2 

Direct  Attack  Moving  Target  Capability  (DAMTC) 

OT  Report 

June  2012 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  (IFF)  System 

OT  Report 

July  2012 

AH-64D  Apache  Block  III  (AB3)  Attack  Helicopter  with  Classified  Annex 

Combined  OT/LFT 

August  201 2 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 

OT  Report 

August  201 2 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2  with  Classified  Annex 

OT  Report 

September  201 2 

Common  Remotely  Operated  Weapon  Station  (CROWS) 

OT  Report 

September  201 2 

Early  Fielding  Reports 

Mk  54  Lightweight  Torpedo  with  Block  Upgrade  (BUG)  Software 

OT  Report 

January  2012 

Massive  Ordnance  Penetrator  (MOP) 

OT  Report 

April  2012 

FOT&E  Reports 

Virginia  Class  Submarine  Low  Frequency  Active  (LFA)  (ACCM) 

OT  Report 

November  201 1 

Tomahawk  Land  Attack  Missile 

OT  Report 

February  2012 

Verification  of  Correction  of  Deficiencies  Report  on  the  Multi-functional  Information 
Distribution  System  (MIDS)  Joint  Tactical  Radio  System  (JTRS) 

OT  Report 

March  2012 

EA-1 8G  Airborne  Electronic  Attack  (AEA)  Aircraft 

OT  Report 

March  2012 

F/A-1 8E/F  Super  Hornet 

OT  Report 

April  2012 

Combined  MH-60R  Multi-Mission  Helicopter  and  the  MH-60S  Multi-Mission  Combat 
Support  Helicopter  Preplanned  Product  Improvement  (P3I)  Program 

OT  Report 

April  2012 

LFT&E  Reports 

Family  of  Medium  Tactical  Vehicles  (FMTV)  A1 P2 

LFT  Report 

October  201 1 

Enhanced  Combat  Helmet  (ECH) 

LFT  Report 

May  201 2 

MAIS  Reports 

EProcurement  System 

OT  Report 

June  2012 

Special  Reports 

Active  Protection  Systems  (APS)  Live  Fire  Test  and  Evaluation 

LFT  Report 

February  2012 

Assessment  of  Department  of  Defense  (DoD)  Information  Assurance  during  Major 
Combatant  Command  (CCMD)  Service  Exercises 

Information  Assurance 

April  2012 

BMD  Reports 

FY1 1  Assessment  of  the  Ballistic  Missile  Defense  System  (BMDS)  (includes  classified 
appendices  A,  B,  C) 

Annual  Report 

February  2012 
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OTHER  FY12  DOT&E  REPORTS 

Program 

Report  Type 

Date 

Operational  Assessment  Reports 

MQ-9  Unmanned  Aerial  System  (UAS) 

OT  Report 

October  201 1 

B-2  Extremely  High  Frequency  (EHF)  Satellite  Communications  (SATCOM)  and  Computer  Upgrade 
Increment  1 

OT  Report 

October  201 1 

AN/AAR-47  Missile  Warning  Set  (MWS)  Hostile  Fire  Indication  (HFI)  Software  Upgrade,  Operational 
Flight  Program  (OFP)  30.41 

OT  Report 

October  201 1 

Key  Management  Infrastructure  (KMI)  Spiral  1 

OT  Report 

October  201 1 

Surveillance  Towed  Array  Sensor  System  (SURTASS)  with  Compact  Low  Frequency  Active  (CLFA) 

OT  Report 

October  201 1 

Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Engineer  Squad  Vehicle  (ESVV) 

Combined  OT/LFT 

October  201 1 

Family  of  Medium  Tactical  Vehicles  (FMTV)  A1 P2 

Combined  OT/LFT 

October  201 1 

Stryker  Double-V  Hull  (DVH  )  Configuration  of  the  Infantry  Carrier  Vehicle  (ICVV)  Scout 

Combined  OT/LFT 

January  2012 

Network  Integration  Evaluation  (NIE)  1 1 .2  with  Classified  Annex 

OT  Report 

January  2012 

Distributed  Common  Ground  System  -  Army  (DCGS-A)  Software  Baseline  (DSB)  1 .0 

OT  Report 

January  2012 

Block  Cycle  Change  03  (BCC  03)  for  the  C-5  Avionics  Modernization  Program  (AMP) 

OT  Report 

February  2012 

C-1 7  Formation  Flight  System  (FFS) 

OT  Report 

February  2012 

C-130J  Station  Keeping  Equipment  (SKE)  Software  Enhancement  (SSE) 

OT  Report 

February  2012 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN)  Increment  2,  Spiral  1 

OT  Report 

March  2012 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (UAS) 

OT  Report 

April  2012 

Department  of  Defense  Teleport  System,  Generation  Three  Phase  Two 

OT  Report 

May  2012 

Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Mortar  Carrier  Vehicle  (MCVV) 

Combined  OT/LFT 

May  2012 

MH-60S  Airborne  Mine  Countermeasures  Helicopter  and  AN/AQS-20A  Mine  Detecting  Sonar 

Combined  OT/LFT 

June  201 2 

Visit,  Board,  Search,  and  Seizure  (VBSS)  Joint  Chemical  Agent  Detector  (JCAD) 

OT  Report 

July  2012 

Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Medical  Evacuation  Vehicle  (MEW) 

Combined  OT/LFT 

July  2012 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  Small  Form  Fit  (HMS)  Manpack  Radio  and 
Joint  Enterprise  Network  Manager  (JENM) 

OT  Report 

July  2012 

Joint  Tactical  Radio  System  (JTRS)  AN/PRC-154  Rifleman  Radio  and  Soldier  Radio  Waveform 

Network  Manager  (SRWNM) 

OT  Report 

August  2012 

Mine  Resistant  Ambush  Protected  (MRAP)  Family  of  Vehicles:  Navistar  Dash  Ambulance  and  MRAP 
All-Terrain  Vehicle  (M-ATV)  Underbody  Improvement  Kit  (UIK) 

Combined  OT/LFT 

August  2012 

MQ-9  Reaper  Block  5  Remotely  Piloted  Aircraft 

OT  Report 

September  201 2 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS)  Increment  1  Release  1 

OT  Report 

September  201 2 

FOT&E  Reports 

BAE-Tactical  Vehicle  System  (TVS)  Caiman  Mine  Resistant  Ambush  Protected  (MRAP)  Vehicle 

LFT&E  Report 

June  201 2 

LFT&E  Reports 

Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Commander's  Vehicle  (CW) 

LFT&E  Report 

January  201 2 

MAIS  Reports 

Battle  Control  System  -  Fixed  (BCS-F)  Increment  3.1 

OT  Report  Update  1 

October  201 1 

Joint  Mission  Planning  System  -  Expeditionary  (JMPS-E) 

IOT&E  Report 

October  201 1 

Public  Key  Infrastructure  (PKI)  Increment  2 

IOT&E  Report 

January  2012 

Global  Command  and  Control  System  -  Joint  (GCCS-J)  Version  4.2.0.9 

OT  Report 

January  2012 

Joint  Mission  Planning  System  (JMPS)  E-8  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS) 
Mission  Planning  Environment  (MPE) 

IOT&E  Report 

February  2012 

Battle  Control  System  -  Fixed  (BCS-F)  Increment  3,  Release  3.1  (R3.1 ) 

OT  Report  Update  2 

May  2012 

Special  Reports 

Hellfire  Romeo  Missile 

LFT&E  Report 

November  201 1 

Stryker  Mobile  Gun  System  (MGS)  Engineering  Change  Order  (ECO)  Block  3 

Combined  OT/LFT 

January  2012 

Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile  (SLAMRAAM) 

OT  Report 

May  2012 
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Program  Oversight 


DOT&E  is  responsible  for  approving  the  adequacy  of  plans  for 
operational  test  and  evaluation  and  for  reporting  the  operational 
test  results  for  all  major  defense  acquisition  programs  to  the 
Secretary  of  Defense,  Under  Secretary  of  Defense  (Acquisition, 
Technology  and  Logistics),  Service  Secretaries,  and  Congress. 
For  DOT&E  oversight  purposes,  major  defense  acquisition 
programs  were  defined  in  the  law  to  mean  those  programs 
meeting  the  criteria  for  reporting  under  Section  2430,  Title  10, 
United  States  Code  (U.S.C.)  (Selected  Acquisition  Reports 
(SARs)).  The  law  (Section  139(a)(2)(B))  also  stipulates  that 
DOT&E  may  designate  any  other  programs  for  the  purpose 
of  oversight,  review,  and  reporting.  With  the  addition  of  such 
“non-major”  programs,  DOT&E  was  responsible  for  oversight  of 
a  total  of  327  acquisition  programs  during  FY 12. 

Non-major  programs  are  selected  for  DOT&E  oversight  after 
careful  consideration  of  the  relative  importance  of  the  individual 
program.  In  determining  non-SAR  systems  for  oversight, 
consideration  is  given  to  one  or  more  of  the  following  essential 
elements: 

•  Congress  or  OSD  agencies  have  expressed  a  high  level  of 
interest  in  the  program. 

•  Congress  has  directed  that  DOT&E  assess  or  report  on  the 
program  as  a  condition  for  progress  or  production. 

•  The  program  requires  joint  or  multi-Service  testing  (the  law 
(Section  139(b)(4))  requires  DOT&E  to  coordinate  “testing 
conducted  jointly  by  more  than  one  military  department  or 
defense  agency”). 

•  The  program  exceeds  or  has  the  potential  to  exceed  the  dollar 
threshold  definition  of  a  major  program  according  to  DoD 
5000.1,  but  does  not  appear  on  the  current  SAR  list  (e.g., 
highly  classified  systems). 


•  The  program  has  a  close  relationship  to  or  is  a  key  component 
of  a  major  program. 

•  The  program  is  an  existing  system  undergoing  major 
modification. 

•  The  program  was  previously  a  SAR  program  and  operational 
testing  is  not  yet  complete. 

This  office  is  also  responsible  for  the  oversight  of  LFT&E 
programs,  in  accordance  with  10  U.S.C  139.  DoD  regulation 
uses  the  term  “covered  system”  to  include  all  categories  of 
systems  or  programs  identified  in  10  U.S.C.  2366  as  requiring 
LFT&E.  In  addition,  systems  or  programs  that  do  not  have 
acquisition  points  referenced  in  10  U.S.C.  2366,  but  otherwise 
meet  the  statutory  criteria,  are  considered  “covered  systems”  for 
the  purpose  of  DOT&E  oversight. 

A  covered  system,  for  the  purpose  of  oversight  for  LFT&E, 
has  been  determined  by  DOT&E  to  meet  one  or  more  of  the 
following  criteria: 

•  A  major  system,  within  the  meaning  of  that  term  in 
10  U.S.C.  2302(5),  that  is: 

-  User-occupied  and  designed  to  provide  some  degree  of 
protection  to  the  system  or  its  occupants  in  combat 

-  A  conventional  munitions  program  or  missile  program 

•  A  conventional  munitions  program  for  which  more  than 
1,000,000  rounds  are  planned  to  be  acquired. 

•  A  modification  to  a  covered  system  that  is  likely  to  affect 
significantly  the  survivability  or  lethality  of  such  a  system. 

DOT&E  was  responsible  for  the  oversight  of  127  LFT&E 
acquisition  programs  during  FY12. 
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Programs  Under  DOT&E  Oversight 
Fiscal  Year  2012 

(As  taken  from  the  September  2012  DOT&E  Oversight  List) 


DoD  PROGRAMS 


AC-130J 

Ballistic  Missile  Defense  System  (BMDS)  Program 
Ballistic  Missile  Technical  Collection  (BMTC) 

Chemical  Demilitarization  -  Chemical  Materials  Agency  (Army 
Executing  Agent)  (CHEM  DEMIL-CMA) 

Chemical  Demilitarization  Program  -  Assembled  Chemical  Weapons 
Alternatives  (CHEM  DEMIL-ACWA) 

Conventional  Prompt  Global  Strike 
Defense  Agency  Initiative  (DAI) 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS) 
Increment  1 

Defense  Readiness  Reporting  System  -  Strategic 

Defense  Security  Assistance  Management  System  (DSAMS)  -  Block  3 

EProcurement 

Global  Combat  Support  System  -  Joint  (GCSS-J) 

Global  Command  and  Control  System  -  Joint  (GCCS-J) 

Integrated  Electronic  Health  Record  (iEHR) 

Joint  Aerial  Layer  Network 

Joint  Biological  Standoff  Detection  System  (JBSDS) 

Joint  Biological  Tactical  Detection  System  (JBTDS) 

Joint  Chemical  Agent  Detector  (JCAD) 

Joint  Command  and  Control  Capabilities  (JC2C)  [Encompasses 
GCCS-FoS  (GCCS-J,  GCCS-A,  GCCS-M),TBMCS-FL,  DCAPES,  GCCS-AF, 
USMC  JTCW,  USMCTCO] 


Joint  Tactical  Radio  System  (JTRS)  Airborne  and  Maritime/Fixed 
Station  (AMF) 

Joint  Tactical  Radio  System  (JTRS)  Enterprise  Network  Manager  (JENM) 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and  Small 
Form  Fit  (HMS)  Radios 

Joint  Tactical  Radio  System  (JTRS)  Network  Enterprise  Domain  (NED) 
Joint  Warning  and  Reporting  Network  (JWARN) 

Key  Management  Infrastructure  (KMI)  Increment  2 
Mid-Tier  Networking  Vehicle  Radio 

Multi-Functional  Information  Distribution  System  (MIDS)  (includes  all 
current  and  planned  integrations  of  MIDS  JTRS  into  USAF  and  USN 
aircraft:  F/A-1 8  E/F,  E-2D,  E-8,  RC-1 35,  EC-1 30  (all  applicable  series 
designations)) 

Next  Generation  Diagnostic  System 
Public  Key  Infrastructure  (PKI)  Increment  2 
Soldier  Radio  Waveform  (SRW)  Network  Manager 

Special  Operations  Command  Dry  Combat  Submersible  Medium 
(DCSM) 

Special  Operations  Command  Next  Generation  Dry  Deck  Shelter 
Teleport,  Generation  III 

Theater  Medical  Information  Program  -  Joint  (TMIP-J)  Block  2 
Virtual  Interactive  Processing  System  (VIPS) 


ARMY  PROGRAMS 


.300  Winchester  Magnum  Mk248  Mod  1  ammunition 
25  mm  Individual  Semi-Automatic  Airburst  System  (ISAAS) 

Abrams  Tank  Modernization  (Ml  E3) 

Abrams  Tank  Upgrade  (Ml  A1  SA/  Ml  A2  SEP) 

Apache  Block  III  (AB3) 

Armed  Aerial  Scout  (previously  named  ARH  Armed  Recon  Helicopter) 
Armored  Multi-Purpose  Vehicle  (AMPV) 

Armored  Truck  -  Heavy  Dump  Truck  (HDT) 

Armored  Truck  -  Heavy  Equipment  Transporter  (HET) 

Armored  Truck  -  Heavy  Expanded  Mobility  Tactical  Truck  (HEMTT) 


Armored  Truck  -  M915A5  Line  Hauler 
Armored  Truck  -  M939  General  Purpose  Truck 
Armored  Truck  -  Palletized  Loading  System  (PLS) 

Army  Integrated  Air  and  Missile  Defense  (AIAMD) 

Army  Vertical  Unmanned  Aircraft  System 
Biometrics  Enabling  Capability  (BEC)  Increment  1 
Black  Hawk  Utility  Helicopter  (UH-60M)  Upgrade  Program 
Bradley  Fighting  Vehicle  System  Upgrade 
Bradley  Modernization  (M2A3  V2) 

Cartridge,  7.62  mm,  M80A1 
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ARMY  PROGRAMS  (continued) 


CH-47F  -  Cargo  Helicopter 

Common  Infrared  Countermeasures  (CIRCM) 

Common  Remotely  Operated  Weapon  Station  (CROWS)  III 
Distributed  Common  Ground  System  -  Army  (DCGS-A) 

Enhanced  AN/TPQ-36  Radar  System  (EQ-36) 

Excalibur  -  Family  of  Precision,  1 55  mm  Projectiles 
FMTV  -  Family  of  Medium  Tactical  Vehicles 

Force  XXI  Battle  Command  Brigade  and  Below -Joint  Capability 
Release  (FBCB2-JCR) 

Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2)  Program 
General  Fund  Enterprise  Business  System  (GFEBS) 

Global  Combat  Support  System  -  Army  (GCSS-Army) 

Ground  Combat  Vehicle  (GCV) 

Guided  Multiple  Launch  Rocket  System  (GMLRS)  -  Alternate  Warhead 
(AW) 

Guided  Multiple  Launch  Rocket  System  (GMLRS)  -  Dual  Purpose 
Improved  Conventional  Munitions  (DPICM) 

Guided  Multiple  Launch  Rocket  System  (GMLRS)  -  Unitary 
Hellfire  Romeo 

High  Mobility  Artillery  Rocket  System  (HIMARS) 

High  Mobility  Multi-purpose  Wheeled  Vehicle  (HMMWV) 

Hostile  Fire  Detection  System 
Individual  Carbine 

Integrated  Personnel  and  Pay  System  -  Army  (Army  IPPS) 

Interceptor  Body  Armor 

Javelin  Anti-Tank  Missile  System  -  Medium 

Joint  Air-to-Ground  Missile  (JAGM) 

Joint  Assault  Bridge 

Joint  Battle  Command  Platform  (JBC-P) 

Joint  Cooperative  Target  Identification  -  Ground  (JCTI-G) 

Joint  Future  Theater  Lift  Concept  (JFTLC) 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor 
(JLENS)  System 

Joint  Lightweight  Tactical  Vehicle  (JLTV) 

Joint  Personnel  Identification  (JPIv2) 

Kiowa  Warrior  Upgrade 

Land  Warrior  -  Integrated  Soldier  Fighting  System  for  Infantrymen 
Light  Utility  Helicopter  (LUH) 

Logistics  Modernization  Program  (LMP) 

Long  Endurance  Multi-Intelligence  Vehicle  (LEMV) 

Ml  200  Knight  Targeting  Under  Armor  (TUA) 


M270A1  Multiple  Launch  Rocket  System  (MLRS) 

M829E4 

Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5  (all  development 
and  integration  programs) 

Modernized  Expanded  Capacity  Vehicle  (MECV)  -  Survivability 
Demonstration 

MQ-1 C  Unmanned  Aircraft  System  Gray  Eagle 
Nett  Warrior 

One  System  Remote  Video  Terminal  (OSRVT) 

Paladin/Field  Artillery  Ammunition  Supply  Vehicle  (FAASV)  Integrated 
Management  (PIM) 

Patriot  Advanced  Capability  3  (PAC-3)  (Missile  only) 

Patriot/Medium  Extended  Air  Defense  System  (MEADS) 

RQ-1 1 B  Raven  -  Small  Unmanned  Aircraft  System 
RQ-7B  SHADOW  -Tactical  Unmanned  Aircraft  System 
Spider  XM7  Network  Command  Munition 

Stryker  Ml  126  Infantry  Carrier  Vehicle  including  Double-V  Hull  Variant 
Stryker  Ml  127  Reconnaissance  Vehicle 
Stryker  Ml  128  Mobile  Gun  System 

Stryker  Ml  1 29  Mortar  Carrier  including  the  Double-V  Hull  Variant 

Stryker  Ml  1 30  Commander's  Vehicle  including  the  Double-V  Hull 
Variant 

Stryker  Ml  131  Fire  Support  Vehicle  Including  the  Double-V  Hull 
Variant 

Stryker  Ml  1 32  Engineer  Squad  Vehicle  Including  the  Double-V  Hull 
Variant 

Stryker  Ml  133  Medical  Evacuation  Vehicle  Including  the  Double-V 
Hull  Variant 

Stryker  Ml  1 34  Anti-Tank  Guided  Missile  (ATGM)  Vehicle  Including  the 
Double-V  Hull  Variant 

Stryker  Ml  135  Nuclear,  Biological,  and  Chemical  Reconnaissance 
Vehicle  (NBCRV)  Including  the  Double-V  Hull  Variant 

Stryker  Modernization  Program 

Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile 
(SLAMRAAM) 

Tactical  Edge  Network  -  Extension 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  1 
Warfighter  Information  Network  -Tactical  (WIN-T)  Increment  2 
Warfighter  Information  Network -Tactical  (WIN-T)  Increment  3 
Warfighter  Information  Network  -Tactical  (WIN-T)  Increment  4 
XM1 1 56  Precision  Guidance  Kit  (PGK) 

XM395  Accelerated  Precision  Mortar  Initiative  (APMI) 
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Acoustic  Rapid  Commercial  Off-the-Shelf  (COTS)  Insertion  (A-RCI)  for 
SONAR 

Advanced  Airborne  Sensor 

Advanced  Extremely  High  Frequency  (AEHF)  Navy  Multiband  Terminal 
(NMT)  Satellite  Program 

Aegis  Modernization 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 

AH-1Z 

AIM-9X  Air-to-Air  Missile  Upgrade  Block  II 
Air  and  Missile  Defense  Radar  (AMDR) 

Air  Warfare  Ship  Self-Defense  Enterprise 

Airborne  Laser  Mine  Detection  System  (AN/AES-1)  (ALMDS) 

Airborne  Mine  Neutralization  System  (AN/ASW-235)  (AMNS) 
Amphibious  Assault  Vehicle  Upgrade 
Amphibious  Combat  Vehicle  (ACV) 

AN/APR-39  Radar  Warning  Receiver 

AN/AQS-20A  Minehunting  Sonar 

AN/BLQ-1 0  Submarine  Electronics  Support  Measures 

AN/BVY-1  Integrated  Submarine  Imaging  System 

AN/SQQ-89A(V)  Integrated  USW  Combat  Systems  Suite 

Anti-Torpedo  Torpedo  Defense  System  including  all  associated 
programs  (Countermeasure  Anti-Torpedo  (CAT),  Torpedo  Warning 
System  (TWS),  and  SLQ-25X  (NIXIE)) 

AR/LSB  -  Airborne  Resupply/Logistics  for  Seabasing 

Assault  Breaching  System  Coastal  Battlefield  Reconnaissance  and 
Analysis  System  Block  II 

Broad  Area  Maritime  Surveillance  (BAMS)  Unmanned  Aircraft  System 
BYG-1  Fire  Control  (Weapon  Control  andTMA) 

CH-53K-  Heavy  Lift  Replacement  Program 
Close-In  Weapon  System  (CIWS)  including  SEARAM 
Cobra  Judy  Replacement  -  Ship-based  Radar  System 
Common  Aviation  Command  and  Control  System  (CAC2S) 
Consolidated  Afloat  Networks  and  Enterprise  Services  (CANES) 
Cooperative  Engagement  Capability  (CEC) 

Countermeasure  Anti-Torpedo 

CVN-78  -  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 

CVN-78  -  Electro-Magnetic  Aircraft  Launching  System 

DDG-1 000  -  Zumwalt  Class  Destroyer  -  includes  all  supporting  PARMs 
and  the  lethality  of  the  LRLAP,  57  mm  and  30  mm  ammunition 

DDG-51  -Arleigh  Burke  Class  Guided  Missile  Destroyer- includes  all 
supporting  PARMs 

DDG-51  Flight  III  -Arleigh  Burke  Class  Guided  Missile 
Destroyer  -  includes  all  supporting  PARMs 

Department  of  Navy  Large  Aircraft  Infrared  Countermeasures  (DoN 
LAIRCM)  Program 


Distributed  Common  Ground  System  -  Marine  Corps  (DCGS-MC) 
Distributed  Common  Ground  System  -  Navy  (DCGS-N) 

E-2D  Advanced  Hawkeye 

EA-1 8G  -  Airborne  Electronic  Attack  Variant  of  the  F/A-1 8  Aircraft 
Enhanced  Combat  Helmet  (ECH) 

Evolved  SeaSparrow  Missile  (ESSM) 

Evolved  SeaSparrow  Missile  Block  2 

F/A-1 8E/F  -  Super  Hornet  Naval  Strike  Fighter 

Future  Pay  and  Personnel  Management  Solution  (FPPS) 

Global  Combat  Support  System  -  Marine  Corps  (GCSS-MC) 

Global  Command  and  Control  System  -  Maritime  (GCCS-M) 
Ground/AirTask  Oriented  Radar  (G/ATOR) 

Infrared  Search  and  Track  System 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  (All  Blocks) 
Joint  and  Allied  Threat  Awareness  System  (JATAS) 

Joint  Expeditionary  Fires 
Joint  High  Speed  Vessel  (JHSV) 

Joint  Precision  Approach  and  Landing  System  (JPALS)  Increment  1 
(Ship  system) 

Joint  Precision  Approach  and  Landing  System  (JPALS)  Increment  2 
(Land  system) 

Joint  Standoff  Weapon  C-1  Variant  (JSOW  C-1 ) 

KC-1 30J  with  Harvest  Hawk 

Landing  Ship  Dock  Replacement  (LSD(X)) 

LHA  6  -  America  Class  Amphibious  Assault  Ship  -  includes  all 
supporting  PARMs 

LHD  8  Amphibious  Assault  Ship 
Light  Armored  Vehicle  (LAV) 

Light  Weight  Tow  Torpedo  Countermeasure  (part  of  LCS  ASW  Mission 
Module) 

Littoral  Combat  Ship  (LCS)  -  includes  all  supporting  PARMs,  and 
57  mm  lethality 

Littoral  Combat  Ship  (LCS)  Mission  Modules  including  30  mm  and 
missile  lethality 

Littoral  Combat  Ship  (LCS)  Surface-to-Surface  Missile  Module 
(follow-on  to  the  interim  Griffin  Missile) 

Littoral  Combat  Ship  (LCS)  Variable  Depth  Sonar  (VDS) 

Logistics  Vehicle  System  Replacement 

LPD-1 7  -  San  Antonio  Class  Amphibious  Transport  Dock 
Ship  -  includes  all  supporting  PARMs  and  30  mm  lethality 

Marine  Personnel  Carrier 

Maritime  Prepositioning  Force  (Future)  Mobile  Landing  Platform 

Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5  (all  development 
and  integration  programs) 

Medium  Tactical  Vehicle  Replacement  (MTV)  Program  (USMC) 
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Medium-Range  Maritime  Unmanned  Aircraft  System 
MH-60R  Multi-Mission  Helicopter  Upgrade 
MH-60S  Multi-Mission  Combat  Support  Helicopter 

Mine  Resistant  Ambush  Protected  (MRAP)  Family  of  Vehicles 
(FoV)  -  including  Special  Operations  Command  vehicles 

Mk  54 Torpedo/Mk  54  Vertical-Launch  Anti-Submarine  (VLA)/MK54 
Upgrades  Including  High-Altitude  Anti-Submarine  Warfare  (ASW) 
Weapon  Capability  (HAAWC) 

Mk48  CBASS  Torpedo 

Mk  48 Torpedo  Mods 

Mobile  User  Objective  System  (MUOS) 

Multi-static  Active  Coherent  (MAC)  System  CNO  project  1 758 
MV-22  Osprey  -  Joint  Advanced  Vertical  Lift  Aircraft 
Naval  Integrated  Fire  Control  -  Counter  Air  (NIFC-CA) 

Navy  Enterprise  Resource  Planning  (ERP) 

Navy  Unmanned  Carrier  Launched  Airborne  Surveillance  and  Strike 
(UCLASS)  System 

Next  Generation  Enterprise  Network  (NGEN) 

Next  Generation  Jammer  (NGJ) 

Offensive  Anti-Surface  Warfare 

Offensive  Anti-Surface  Warfare  Interim  Capability  (also  called 
Tomahawk  Offensive  Anti-Surface  Warfare  Interim  Capability) 

Ohio  Replacement  Program  (Sea-based  Strategic 
Deterrence)  -  including  all  supporting  PARMs 

Organic  Airborne  and  Surface  Influence  Sweep  (OASIS) 

P-8A  Poseidon  Program 

Rapid  Airborne  Mine  Clearance  System  (RAMICS) 

Remote  Minehunting  System  (RMS) 

Replacement  Oiler 

Rolling  Airframe  Missile  (RAM)  including  RAM  Block  1 A  Helicopter 
Aircraft  Surface  (HAS)  and  RAM  Block  2  Programs 


Sea-Based  Support  to  Special  Forces 
Ship  Self-Defense  System  (SSDS) 

Ship-to-Shore  Connector  (SSC) 

Small  Tactical  Unmanned  Aerial  System  (STUAS)  -  UAS  Tier  II 

SSN  774  Virginia  Class  Submarine 

SSN  784  Virginia  Class  Block  III  Submarine 

Standard  Missile-2  (SM-2)  including  all  mods 

Standard  Missile-6  (SM-6) 

Submarine  Torpedo  Defense  System  (SubTDS)  including 
countermeasures  and  Next  Generation  Countermeasure  (NGCM) 
System 

Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  2 
Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  3 
Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  4 

Surface  Mine  Countermeasures  Unmanned  Undersea  Vehicle  (also 
called  Knifefish  UUV)  (SMCM  UUV) 

Surveillance  Towed  Array  Sonar  System/Low  Frequency  Active 
(SURTASS/LFA)  including  Compact  LFA  (CLFA) 

Tactical  Tomahawk  -  Follow-on  to  Tomahawk  Baseline  Missile 

T-AKE  -  Lewis  &  Clark  C lass  of  Auxiliary  Dry  Cargo  Ships 
(T-AKE)  -  includes  all  supporting  PARMs 

Torpedo  Warning  System  (Previously  included  with  Surface  Ship 
Torpedo  Defense  System)  including  all  sensors  and  decision  tools 

Trident  II  Missile  -  Sea  Launched  Ballistic  Missile 
UH-1Y 

Unmanned  Influence  Sweep  System  (UISS)  include  Unmanned 
Surface  Vessel  (USV)  and  Unmanned  Surface  Sweep  System  (US3) 

Unmanned  Undersea  Vehicle  Program 

Vertical  Take-off  and  LandingTactical  Unmanned  AirVehicle  (VTUAV) 
(Fire  Scout) 

VXX  -  Presidential  Helicopter  Fleet  Replacement  Program 


AIR  FORCE  PROGRAMS 


20  mm  PGU-28/B  Replacement  Combat  Round 
Advanced  Extremely  High  Frequency  (AEHF)  Satellite  Program 
Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) 
Advanced  PilotTrainer 

Air  Force  Distributed  Common  Ground  System  (AF-DCGS) 

Air  Force  Integrated  Personnel  and  Pay  System  (AF-IPPS) 

Air  Force  Intranet  (AFNet)  Combat  Information  Transport  System 
(CITS)  Migration  Urgent  Operational  Need 

Air  Force  Intranet  (AFNet)  Increment  1 
Air  Force  Intranet  (AFNet)  Increment  2 


Air  Force  Intranet  (AFNet)  Modernization  capabilities  (Bitlocker,  Data 
at  Rest,  Situational  Awareness  Modernization) 

Air  Operations  Center  -  Weapon  System  (AOC-WS)  Initiatives  10.0  and 
10.1 

Air  Operations  Center -Weapon  System  (AOC-WS)  Initiative  10.2 
Airborne  Signals  Intelligence  Payload  (ASIP)  Family  of  Sensors 
Airborne  Warning  and  Control  System  (AWACS)  Block  40/45  Upgrade 
ALR-69A  RadarWarning  Receiver 

B-2  Advanced  Extremely  High  Frequency  (EHF)  Satellite 
Communications  (SATCOM)  and  Computer  Capability  Increment  I 
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B-2  Advanced  Extremely  High  Frequency  (EHF)  Satellite 
Communications  (SATCOM)  and  Computer  Capability  Increment  II 

B-2  Defensive  Management  System  Modernization  (DMS) 

B-61  Mod  12  Life  Extension  Program 

Battle  Control  System  -  Fixed  (BCS-F)  3.1 

Battle  Control  System  -  Fixed  (BCS-F)  3.2 

C-1 30  Aircraft  Avionics  Modernization  Program  (AMP) 

C-1 30J  -  Hercules  Cargo  Aircraft  Program 

C-1 7A  -  Globemaster  III  Advanced  Cargo  Aircraft  Program 

C-27J  Joint  Cargo  Aircraft  (JCA) 

C-5  Aircraft  Avionics  Modernization  Program  (AMP) 

C-5  Aircraft  Reliability  Enhancement  and  Re-Engining  Program  (RERP) 
C-5  Core  Mission  Computer  and  Weather  Radar  Replacement 
Cobra  Judy  Replacement  Mission  Planning  Tool 

Command  and  Control  Air  Operations  Suite  (C2AOS)/Command 
and  Control  Information  Services  (C2IS)  (Follow-on  to  Theater  Battle 
Management  Core  Systems) 

CV-22  Osprey  -  Joint  Advanced  Vertical  Lift  Aircraft 

Defense  Enterprise  Accounting  and  Management  System  -  Air  Force 
(DEAMS-AF) 

Enhanced  Polar  System  (EPS) 

Evolved  Expendable  Launch  Vehicle 
Expeditionary  Combat  Support  System  (ECSS) 

F-1 5E  Radar  Modernization  Program 

F-22  -  Raptor  Advanced  Tactical  Fighter 

F-35  -  Lightning  II  Joint  Strike  Fighter  (JSF)  Program 

Family  of  Beyond  Line-of-Sight  Terminals  (FAB-T) 

Family  of  Beyond  Line-of-Sight  Terminals  (FAB-T),  Increment  2  (High 
Data  Rate  Airborne  Terminal) 

Full  Scale  Aerial  Target 
Global  Broadcast  Service  (GBS) 

Global  Broadcast  Service  (GBS)  Defense  Enterprise  Computing  Center 
(DECC) 

Global  Hawk  (RQ-4B)  Block  30  -  High  Altitude  Endurance  Unmanned 
Aircraft  System 

Global  Hawk  (RQ-4B)  Block  40  -  High  Altitude  Endurance  Unmanned 
Aircraft  System 

Global  Positioning  Satellite  (GPS)  Next  Generation  Control  Segment 
(OCX) 

Global  Positioning  Satellite  III  (GPS-IIIA) 

HC/MC-130  Recapitalization 


HH-60G/CRH  (Combat  Rescue  Helicopter) 

Information  Transport  System  (ITS)  Increment  2 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN) 

Increment  2 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN) 

Increment  4 

Joint  Air-to-Surface  Standoff  Missile  (JASSM)  and  JASSM  -  Extended 
Range  (JASSM-ER)) 

Joint  Direct  Attack  Munition  (JDAM) 

Joint  Mission  Planning  Systems  (JMPS)  -  Air  Force  (including  RC-1 35, 
E-8/E-3,  F-22,  A- 10) 

Joint  Space  Communication  Layer 

Joint  Space  Operations  Center  (JSpOC)  Mission  System  (JMS) 

KC-46  -Tanker  Replacement  Program 

Large  Aircraft  Infrared  Countermeasures  (LAIRCM)  Program 

Long  Range  Stand  Off  (LRSO)  Weapon 

Long  Range  Strike  Bomber 

Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5  (all  development 
and  integration  programs) 

Massive  Ordnance  Penetrator  (MOP) 

Military  GPS  User  Equipment  (MGUE) 

Miniature  Air-Launched  Decoy  (MALD) 

Miniature  Air-Launched  Decoy  -  Jammer  (MALD-J) 

MQ-9  Reaper  -  Unmanned  Aircraft  System 
MQ-X 

Multi-Platform  RadarTechnology  Insertion  Program  (MP  RTIP) 

Navstar  Global  Positioning  System  (GPS)  (includes  Satellites,  Control, 
and  User  Equipment) 

Nuclear  Detection  (NUDET)  System  (NDS) 

Presidential  Aircraft  Recapitalization  (PAR)  Program  -  Air  Force  One 
Recapitalization  Program 

Small  Diameter  Bomb,  Increment  I 
Small  Diameter  Bomb,  Increment  II 
Space  Fence  (SF) 

Space-Based  Infrared  System  (SBIRS)  Program,  High  Component 
Space-Based  Space  Surveillance  (SBSS)  Block  10  Follow-on 
Three-dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 
Vulnerability  Lifecycle  Management  System  (VLMS)  1 .5 
Weather  Satellite  Follow-on  (WSF) 

Wideband  Global  Satellite  Communications  (SATCOM)  (WGS)  Program 
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Problem  Discovery  Affecting  Operational  Test  and  Evaluation 


One  purpose  of  test  and  evaluation  is  to  determine  if  thresholds  in 
the  approved  Capability  Production  Document  (CPD)  have  been 
satisfied.  The  Acquisition  Executive  needs  this  information  in 
making  production  decisions,  but  satisfying  these  measures  is  often 
not  equivalent  to  achieving  the  required  combat  capability  needed 
for  mission  accomplishment.  A  comprehensive  evaluation  of 
operational  effectiveness,  operational  suitability,  and  survivability 
provides  the  Acquisition  Executive  and  operational  users  with 
information  regarding  a  system’s  combat  capability.  This 
evaluation  can  only  be  done  after  operational  testing  (OT)  under 
realistic  combat  conditions,  which  includes  end-to-end  testing 
with  operational  users  across  the  intended  operational  envelope 
and  within  the  context  of  the  system-of-systems  in  which  it  will 
operate. 

The  Deputy  Assistant  Secretary  of  Defense  (DASD) 

Developmental  Test  and  Evaluation  (DT&E)  conducts  an 
assessment  of  all  Major  Defense  Acquisition  Programs  and  special 
interest  programs  prior  to  their  OT;  this  DT&E  assessment  reports 
on  a  system’s  demonstrated  ability  to  meet  its  Key  Performance 
Parameters  and  assesses  the  risk  of  the  system’s  ability  to 
successfully  complete  OT.  The  DT&E  assessment  is  based  on 
capabilities  demonstrated  during  developmental  testing  (DT),  early 
OT,  and  criteria  from  the  Test  and  Evaluation  Master  Plan  and 
requirements  documents.  The  DT&E  community  engages  with 
program  offices  early  and  often  throughout  a  program’s  acquisition 
cycle,  observing  both  contractor  and  government  DT.  The  DT 
and  early  OT  events  provide  the  program  manager  opportunities 
to  discover  and  correct  problems  that  could  prevent  a  system  from 
delivering  its  required  combat  capability.  As  such,  the  test  events 
should  include  as  much  operational  realism  as  possible,  and  also 
include  military  operators  and  maintainers  whenever  possible. 

The  early  test  events  should  also  provide  information  to  the 
requirements  and  resource  sponsors  for  the  system  to  ensure  that 
the  documented  requirements  are  still  relevant  and  feasible.  By 
the  time  of  the  Initial  Operational  Test  and  Evaluation  (IOT&E), 
discovery  of  significant  issues  affecting  combat  capability  should 
be  rare,  and  lingering  problems  from  DT  should  have  been 
resolved. 

Last  year,  I  added  a  new  section  to  my  Annual  Report  assessing 
systems  under  my  oversight  in  2010-2011  with  regard  to 
problem  discovery  during  testing.  My  assessment  fell  into  two 
categories:  systems  with  significant  issues  observed  in  OT  that 


should,  in  my  view,  have  been  discovered  and  resolved  prior  to 
the  commencement  of  OT,  and  systems  with  significant  issues 
observed  during  early  testing  that,  if  not  corrected,  could  adversely 
affect  my  evaluation  of  those  systems’  effectiveness,  suitability, 
and  survivability  during  IOT&E.  This  year,  I  am  providing  an 
update  to  the  status  of  those  systems  identified  last  year,  as  well 
as  my  assessment  of  systems  under  my  oversight  in  2012  within 
those  two  categories. 

Last  year,  I  reported  that  four  of  the  seven  Assessments  of 
Operational  Test  Readiness  (AOTRs)  that  I  received  from  the 
DASD(DT&E)  recommended  that  the  programs  not  proceed  to 
IOT&E,  but  that  the  program  proceeded  anyway.  Regardless 
of  the  AOTR  recommendation,  six  of  those  seven  programs 
experienced  significant  issues  in  their  IOT&Es:  the  C-5 
Reliability  Enhancement  and  Re-Engining  Program  (RERP); 
RQ-4B  Global  Hawk  Blocks  20  and  30;  Standard  Missile-6 
(SM-6);  Multi-functional  Information  Distribution  System 
(MIDS)  Joint  Tactical  Radio  System  (JTRS);  Stryker  Nuclear, 
Biological,  and  Chemical  Reconnaissance  Vehicle  (NBCRV); 
and  Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and 
Small  Form  Fit  (HMS)  Rifleman  Radio. 

1  have  received  12  additional  assessments  from  DASD(DT&E) 
since  my  report  last  year;  these  are  listed  in  the  table  below. 

Of  the  12  reports,  2  recommended  not  proceeding  to  IOT&E: 
MQ-1C  Gray  Eagle  and  JTRS  HMS  Manpack  Radio.  Despite 
the  recommendation,  both  of  these  systems  proceeded  to  IOT&E. 
The  JTRS  HMS  Manpack  Radio  performed  poorly  in  the  IOT&E, 
as  predicted  by  the  AOTR;  however,  the  MQ-1C  Gray  Eagle 
performed  well  in  IOT&E  despite  DT  results  suggesting  poor 
reliability  that  would  affect  the  test  outcome.  In  fact,  the  Gray 
Eagle  IOT&E  demonstrated  that  the  modeling  assumptions  that 
established  the  reliability  requirements  thresholds  were  not  valid. 
As  a  result,  the  Army  is  reassessing  whether  those  reliability 
thresholds  should  be  changed.  Additionally,  as  discussed  in  this 
section  last  year,  the  Warfighter  Information  Network  -  Tactical 
(WIN-T)  Increment  2  had  both  performance  and  reliability  issues 
during  early  testing,  but  these  issues  were  not  assessed  by  the 
DT&E  AOTR.  Two  of  the  systems  listed  below  are  still  in-test: 

P-8  and  Joint  Space  Operations  Center  (JSpOC)  Mission  System 
(JMS)  Increment  1. 


DASD(DT&E)  Assessments  of  Operational  Test  Readiness  (AOTRs) 

AIM-9X  Air-to-Air  Missile  Upgrade 

Joint  Space  Operations  Center  (JSpOC)  Mission  Systems  (JMS)  Increment  1 

Apache  Block  III 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and  Small  Form  Fit 
(HMS)  Manpack  Radio 

B-2  Extremely  High  Frequency  (EHF) 

MQ-1 C  Gray  Eagle  Unmanned  Aircraft  System  (UAS) 

C-1 30  Avionics  Modernization  Program  (AMP) 

P-8 

E-2D  Advanced  Hawkeye 

Space-Based  Infrared  System  (SBIRS) 

HC/MC-130J 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2 
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PROGRESS  UPDATES  ON  DISCOVERIES  REPORTED  LASTYEAR 

Last  year,  I  identified  23  systems  that  had  significant  issues  in  early  testing  that  should  be  corrected  prior  to  IOT&E.  The  following 
table  provides  an  update  on  the  progress  those  systems  made  in  implementing  fixes  to  those  problems. 


FY1 1  DISCOVERIES  IN  EARLY  TESTING  THAT  SHOULD  BE  CORRECTED  PRIOR  TO  IOT&E 

Fixes  Implemented  and 
Demonstrated  in  OT 

Fixes  Implemented; 
Currently  in  OT  or  Planning  OT 

Some  Fixes  Implemented; 
Testing  Constrained  Pending 
Future  Acquisition  Decisions 

No  Fixes  Planned 

Apache  Block  III 

Aegis  Modernization 

Joint  Tactical  Radio  System  (JTRS) 
Handheld,  Manpack,  and  Small 

Form  Fit  (HMS)  Manpack  Radio 

Defense  Enterprise  Accounting  and 
Management  System  (DEAMS) 

EProcurement 

AIM-1 20  Advanced  Medium-Range 
Air-to-Air  Missile  (AMRAAM) 

RQ-4B  Global  Hawk  Block  30 
High-Altitude,  Long-Endurance 
Unmanned  Aerial  System  (UAS) 

LHA-6 

Joint  Tactical  Radio  System  (JTRS) 
Network  Enterprise  Domain  (NED) 

AN/TPQ-53  Radar  (formerly  the 
Enhanced  AN/TPQ-36  Radar  System 
(EQ-36)) 

Vertical  Take-Off  and  Landing 
Unmanned  Aerial  Vehicle  (VTUAV) 

Littoral  Combat  Ship  (LCS)  Mission 
Modules 

MQ-1C  Gray  Eagle  Unmanned 

Aircraft  System  (UAS) 

E-2D  Advance  Hawkeye 

MQ-9  Reaper  Armed  Unmanned 
Aircraft  System  (UAS) 

Spider  XM7  Network  Command 
Munition 

Joint  High  Speed  Vessel  (JHSV) 

Miniature  Air-Launched 

Decoy  -  Jammer  (MALD-J) 

Mk48  Advanced  Capability 
(ADCAP)  Mod  7  Common 

Broadband  Advanced  Sonar  System 
(CBASS) 

Mk  54  Lightweight  Torpedo 

P-8A  Poseidon 

Surveillance  Towed  Array  Sensor 
System  (SURTASS)  with  Compact 

Low  Frequency  Active  (CLFA) 

Warfighter  Information 

Network -Tactical  (WIN-T) 
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Last  year,  I  identified  17  systems  that  had  significant  issues  in  10T&E  that  should  have  been  discovered  and  resolved  prior  to 
commencement  of  operational  testing.  The  following  table  provides  an  update  on  the  status  of  those  systems,  as  well  as  the  progress 
those  systems  have  made  in  implementing  fixes  to  the  problems. 


FY1 1  DISCOVERIES  IN  IOT&ETHAT  SHOULD  HAVE  BEEN  RESOLVED  PRIOR  TO  OPERATIONAL  TEST 

Fixes  Implemented  and 
Demonstrated  in  FOT&E 

Fixes  Implemented; 

But  New  Issues  Discovered 

Fixes  Implemented; 
Currently  in  OT 

No  Fixes  Planned 

C-130J 

AGM-88E  Advanced  Anti-Radiation 
Guided  Missile  (AARGM) 

LPD-17 

Force  XXI  Battle  Command 

Brigade  and  Below  (FBCB2)  Joint 
Capabilities  Release  (JCR) 

Common  Aviation  Command  and 
Control  System  (CAC2S) 

Standard  Missile-6  (SM-6) 

Nett  Warrior 

Stryker  Nuclear,  Biological,  and 
Chemical  Reconnaissance  Vehicle 
(NBCRV) 

CV-22  Osprey 

Vertical  Launch  Anti-Submarine 
Rocket  (VLA)  with  Mk  54  Mod  0 
Lightweight  Hybrid  Torpedo 

Department  of  the  Navy 
(DoN)  Large  Aircraft  Infrared 
Countermeasures  (LAIRCM) 

Financial  Information  Resource 
System  (FIRST) 

Multi-functional  Information 
Distribution  System  (MIDS)  Joint 
Tactical  Radio  System  (JTRS) 

Navy  Multiband  Terminal  (NMT) 

Space-Based  Space  Surveillance 
(SBSS) 

Additionally,  2  of  17  programs  were  cancelled:  Early  Infantry  Brigade  Combat  Team  (E-IBCT)  and  Joint  Tactical  Radio  System  (JTRS)  Ground  Mobile  Radio  (GMR). 
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PROBLEMS  DISCOVERED  DURING  OPERATIONAL  TEST  AND  EVALUATION 
THAT  SHOULD  HAVE  BEEN  DISCOVERED  DURING  DEVELOPMENTAL  TEST  AND  EVALUATION 


SIGNIFICANT  DISCOVERIES  IN  FY12  IOT&E 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 

Key  Management  Infrastructure  (KMI)  Increment  2 

ALR-69  Radar  Warning  Receiver  (RWR) 

Mine  Resistant  Ambush  Protected  (MRAP)  Caiman  Multi-Terrain 
Vehicle  (CMTV) 

Battle  Control  System  -  Fixed  (BCS-F)  Release  3.2 

Mine  Resistant  Ambush  Protected  (MRAP)  Dash  Ambulance 

Distributed  Common  Ground  System  -  Army  (DCGS-A) 

Miniature  Air-Launched  Decoy  -  Jammer  (MALD-J) 

E-2D  Advanced  Hawkeye 

MV-22  Osprey 

E-3  Airborne  Warning  and  Control  System  (AWACS)  Block  40/45 
Upgrade 

Standard  Missile-6  (SM-6) 

Joint  Mission  Planning  System  -  Air  Force  (JMPS-AF)  Mission 
Planning  Environment  (MPE)  E-8 

Virginia  Class  Submarine  Modernized  with  the  APB-09  Acoustic 
Rapid  Commercial  Off-the-Shelf  (COTS)  Insertion  (A-RCI)  Sonar 
System  and  AN/BYG-1  Combat  Control  System 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and 

Small  Form  Fit  (HMS)  Manpack  Radio 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and 

Small  Form  Fit  (HMS)  Rifleman  Radio 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 

The  AARGM  program  spent  most  of  F  Y 1 1  correcting  hardware 
and  software  deficiencies  discovered  in  DT  and  during  its  first 
IOT&E  attempt.  Once  IOT&E  began  the  second  time,  the 
Navy  provided  requirements  changes  in  response  to  deficiencies 
identified  since  the  first  IOT&E  attempt  was  terminated,  and 
hence,  the  test  scenarios  were  less  stressing  than  originally 
planned.  Additionally,  new  anomalies  were  discovered: 

•  AARGM  Guidance  Section/Control  Section  communication 
failures  caused  a  significant  number  of  operational  mission 
failures.  The  problem  occurred  during  specific  IOT&E  threat 
scenarios,  but  the  system  deficiency  identified  is  one  that 
should  have  been  identified  with  adequate  DT&E. 

•  A  classified  deficiency  in  performance  required  an  adjusted 
threat  representation. 

ALR-69  Radar  Warning  Receiver  (RWR) 

The  Air  Force  began  operational  flight  testing  in  May  2012, 
knowing  that  the  system  would  likely  not  meet  several  thresholds 
based  on  DT  that  occurred  between  February  and  May  2011. 
Additional  deficiencies  were  observed  in  OT: 

•  Threat  symbol  splitting  (when  one  threat  signal  received  by  the 
system  produces  multiple  threat  symbols  at  different  azimuths 


on  the  cockpit  display)  degraded  the  aircrew’s  situational 
awareness  as  to  which  displayed  threats  are  “real,”  where  those 
real  threats  are  located,  and  inhibited  the  aircrew’s  ability 
to  appropriately  react  to  the  threat) s)  in  a  timely  manner. 

The  threat  symbol  splitting  deficiency  did  not  occur  during 
DT.  The  program  believes  it  was  strictly  a  software  timing 
problem,  and  they  modified  the  software  and  demonstrated  the 
fix  in  the  laboratory  after  the  IOT&E.  No  flight  testing  has 
been  accomplished  to  verify  the  fix. 

Battle  Control  System  -  Fixed  (BCS-F)  Release  3.2 

The  Air  Force  conducted  OT  of  BCS-F  from  April  through 

August  2012,  at  the  System  Support  Facility  (SSF)  and  all  four 

U.S.  operational  air  defense  sectors. 

•  A  critical  deficiency  was  discovered  during  OT  at  the  Eastern 
Air  Defense  Sector.  Random  tracks  were  not  being  passed 
from  the  BCS-F  system  to  the  Joint  Air  Defense  Operations 
Center  at  Bolling  Air  Force  Base,  Washington,  D.C.  This 
deficiency  causes  a  loss  of  situational  awareness  for  the 
operators  conducting  surveillance  of  the  National  Capital 
Region  and  results  in  an  inaccurate  air  picture.  The  problem 
with  the  forwarding  of  tracks  could  not  be  identified  at  the 
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SSF  during  DT&E  since  the  SSF  cannot  replicate  sector  link 
architecture. 

Distributed  Common  Ground  Station  -  Army  (DCGS-A) 

The  Army  conducted  the  DCGS-A  Software  Baseline  1.0  IOT&E 
from  May  through  June  2012  at  Fort  Stewart,  Georgia.  DOT&E 
found  the  system  not  operationally  effective,  not  operationally 
suitable,  and  not  survivable  because  of  deficiencies  identified  in 
the  OT: 

•  Effective  workflow  is  inhibited  for  the  development  of 
intelligence  products  to  support  operations  because  the  system 
configuration  as  tested  placed  the  fusion  capability  in  the 
Secret  Compartmented  Information  (SCI)  (high)  side  even 
though  most  of  the  data  necessary  for  fusion  are  in  the  Secret 
(low)  side.  Additionally,  collection  management  tools  are  on 
the  high  side,  but  collection  managers  need  to  work  closely 
with  the  brigade  operations  staff  on  the  low  side.  Human 
intelligence  tools  are  split  between  the  high  side  and  low  side, 
but  human  intelligence  analysts  manage  and  interview  their 
sources  on  the  low  side.  Developmental  testing  and  Early 
User  Testing  were  conducted  in  a  laboratory  environment  that 
did  not  replicate  the  physical  separation  and  security  barriers 
of  the  deployed  configuration. 

•  The  targeting  software  in  the  SCI  enclave  used  first  known 
location  rather  than  the  last  known  location.  The  DT  showed 
the  target  algorithm  to  be  correct,  but  was  not  robust  enough  to 
discover  this  deficiency. 

•  DCGS-A  was  not  reliable  because  of  a  large  number  of 
software  problems.  The  program  has  not  rigorously  tracked 
metrics  identifying  trends  in  software  maturity,  such  as  the 
number  of  new  software  problems  opened  and  the  number  of 
software  problems  closed. 

E-2D  Advanced  Hawkeye 

The  Navy  conducted  the  E-2D  IOT&E  from  February  to 
September  2012.  The  evaluation  is  currently  ongoing,  but  the 
following  deficiencies  were  revealed: 

•  Cooperative  Engagement  Capability  (CEC)  software 
deficiencies  associated  with  the  CEC  system  generating 
multiple  tracks  for  the  same  contact  were  outstanding  upon 
entering  IOT&E;  thus,  CEC  was  decoupled  from  the  E-2D 
IOT&E.  Corrections  to  the  CEC  system  have  continued 
throughout  2012.  The  system  is  now  in  test.  It  is  likely  that 
current  E-2D  fixes  will  not  address  all  shortfalls  in  the  current 
CEC  system.  Ongoing  work  is  required,  some  of  which  is 
required  for  other  systems  separate  from  E-2D  and  CEC. 

•  Radar  track  re-labeling  was  observed  in  DT,  but  the  full 
magnitude  of  the  problem  only  manifested  itself  under  the 
conditions  of  IOT&E. 

E-3  Airborne  Warning  and  Control  System  (AWACS) 

Block  40/45  Upgrade 

The  Air  Force  conducted  a  24-flight  IOT&E  operating  from  the 
E-3  main  operating  base,  Tinker  Air  Force  Base,  Oklahoma  City, 
Oklahoma,  between  March  and  June  2012.  The  two  operational 


Block  40/45  E-3  aircraft  participated  in  several  large  force 
exercises.  The  test  included  flights  working  with  assets  from 
all  four  Services  in  training  areas  on  both  coasts  as  well  as  over 
land.  The  Block  40/45  AWACS  was  not  ready  to  enter  IOT&E, 
in  addition  to  aircrews  and  maintainers  not  having  representative 
training. 

•  The  mission  planning  system  and  mission  computing  start-up 
checklist  were  never  tested  in  DT&E  and  were  used  for  the 
first  time  in  IOT&E. 

•  The  system  was  designed  to  the  interoperability  standards  in 
place  when  the  development  contract  was  written.  The  aircraft 
does  not  provide  Link  16  capabilities  that  are  equivalent  to  the 
legacy  Block  30/35  it  replaces.  Many  of  the  tactical  datalink 
deficiencies  were  caused  by  the  Air  Force  not  modifying 

the  system  design  to  reflect  changes  in  interoperability 
standards  during  Block  40/45  development.  The  satellite 
communications  terminal  did  not  provide  an  operationally 
useful  capability  to  receive  digital  information. 

Joint  Mission  Planning  System  -  Air  Force  (JMPS-AF)  Mission 
Planning  Environment  (MPE)  E-8 

The  Air  Force  paused  the  IOT&E  of  the  E-8  MPE,  the 
representative  test  platform  for  JMPS-AF  Increment  IV,  in 
September  2011  to  allow  the  Program  Office  to  develop  and 
integrate  corrective  actions  to  deficiencies  identified  during  OT. 
Following  additional  development  and  regression  testing,  the  Air 
Force  certified  E-8  MPE  version  1.3  ready  for  resumed  OT.  The 
Air  Force  intends  to  re-execute  the  entire  IOT&E  in  early  FY13. 
DOT&E’s  assessment  of  the  paused  IOT&E  noted  significant 
deficiencies  that  were  not  identified  during  DT&E: 

•  The  time  needed  for  E-8  MPE  software  installation  was 
lengthy,  due  in  large  part  to  anomalies  in  the  software 
functionality  and  installation  process 

•  Threat  database  information  was  not  easily  accessible  or 
usable;  training  for  intelligence  specialists  was  inadequate 

•  Inability  to  transfer  mission  plans  to  the  aircraft 

•  Critical  calculation  errors  of  the  magnetic  variation  for 
user-specified  waypoints 

•  Could  not  plan  missions  with  in-flight  delays 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and 
Small  Form  Fit  (HMS)  Manpack  Radio 

Although  the  DASD(DT&E)  AOTR  stated  the  Manpack  radio 
was  not  sufficiently  mature  to  enter  the  planned  Multi-Service 
Operational  Test  and  Evaluation  (MOT&E),  the  Army  proceeded 
to  conduct  the  test  as  a  part  of  the  Network  Integration 
Evaluation  (NIE)  12.2.  DOT&E  assessed  the  Manpack  as  not 
operationally  effective  due  to  the  poor  performance  of  the  Single 
Channel  Ground  and  Airborne  Radio  System  (SINCGARS) 
waveform  and  not  operationally  suitable  due  to  a  failure  to  meet 
reliability  or  availability  requirements.  The  Manpack  radio 
AOTR  had  outlined  these  major  MOT&E  deficiencies  prior  to 
OT.  In  September  2012,  the  Army  conducted  a  Government 
Development  Test  (GDT)  3  to  demonstrate  improvements  in 
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MOT&E  deficiencies.  During  GDT  3,  the  Manpack  radio 
demonstrated: 

•  Improved  performance  of  the  SINCGARS  waveform  that 
met  requirements  of  mounted  and  dismounted  transmission 
range,  voice  quality,  and  call  completion  rates  under  benign 
conditions  of  developmental  test. 

•  Poor  reliability  with  the  Solider  Radio  Waveform  (SRW) 
waveform  demonstrating  177  hours  Mean  Time  Between 
Essential  Function  Failure  compared  to  the  Manpack  radio 
requirement  of  477  hours.  This  translates  to  a  66  percent 
chance  of  completing  a  72-hour  mission  compared  to  a 
requirement  of  86  percent. 

Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack,  and 
Small  Form  Fit  (HMS)  Rifleman  Radio 

From  October  to  November  2011,  the  Army  Test  and  Evaluation 
Command  conducted  the  Rifleman  Radio  IOT&E  at  White  Sands 
Missile  Range,  New  Mexico,  as  part  of  the  Army’s  NIE  12.1. 
Operational  units  tested  the  Rifleman  Radio  using  the  Soldier 
Radio  Waveform  Network  Manager  (SRWNM)  to  plan  and  load 
SRW  network  configurations  into  the  radios.  From  February 
through  March  2012,  the  Army  conducted  the  Rifleman  Radio 
GDT  2.3  at  the  Electronic  Proving  Ground  at  Fort  Fluachuca, 
Arizona.  The  Army  conducted  this  GDT  to  complete  DT  that  the 
Army  should  have  completed  prior  to  IOT&E.  In  April  2012,  the 
Army  conducted  a  follow-on  developmental  test,  GDT  2.3a.  The 
Anny  used  this  follow-on  event  to  confirm  fixes  to  deficiencies 
observed  during  GDT  2.3. 

•  The  SRWNM  was  not  employed  with  the  Rifleman  Radio  prior 
to  IOT&E.  The  poor  performance  of  the  SRWNM  adversely 
affected  the  performance  of  the  Rifleman  Radio. 

•  The  software  version  used  in  the  Rifleman  Radio  for  IOT&E 
was  not  the  final  version  to  include  all  the  security  features 
required  by  the  National  Security  Agency  (NSA)  certification. 
The  NSA  requirements  updated  software  caused  numerous 
essential  function  failures  during  GDT  2.3,  which  followed 
IOT&E.  GDT  2.3  reliability  was  so  poor  that  the  Army 
executed  a  GDT  2.3a  to  reassess  DT  reliability  with  installed 
security  fixes.  If  the  DT  had  been  conducted  prior  to  IOT&E, 
the  Army  would  have  produced  a  more  reliable  radio  for 
operational  test. 

•  Problems  with  reliability,  range,  battery  life,  and  thermal 
characteristics  were  found  in  early  OT. 

•  Prior  to  the  IOT&E,  problems  with  the  communications 
security  retention  battery  would  have  negatively  affected 
suitability. 

•  Post-IOT&E,  additional  problems  were  found  with  the 
Rifleman  Radio  including  spontaneous  self-initiated  shutdown, 
failures  to  transmit  and  receive,  and  the  SRW  network  not 
healing  in  a  timely  manner  after  radios  that  had  separated  from 
the  network  rejoined.  These  deficiencies  have  been  fixed  and 
demonstrated  in  DT. 

•  All  deficiencies  have  been  shown  to  be  fixed  or  improving 
(reliability  still  not  met)  but  should  still  be  confirmed  in  a 
formal  GDT  prior  to  the  competitive  IOT&E-2. 


Key  Management  Infrastructure  (KMI)  Increment  2 

The  Joint  Interoperability  Test  Command  conducted  an  IOT&E 
from  July  until  August  2012.  The  results  were  a  marked 
improvement  over  previous  operational  assessments;  however, 
there  were  still  several  operational  effectiveness  and  suitability 
problems  uncovered  during  the  testing  event  that  must  be 
corrected  before  continued  deployment.  The  KMI  program 
and  vendor  regression  testing  of  software  was  problematic  and 
inconsistent.  Lacking  thorough  regression,  software  fixes  in 
newer  releases  often  broke  previously  functioning  components. 

•  OT  identified  some  problems  that  were  missed  by  DT, 
including  problems  with  Electronic  Key  Management  System 
(EKMS)  to  KMI  transition.  High  Assurance  Internet  Protocol 
Encryptor  (KG-250)  configuration,  virtual  private  network 
establishment,  and  data  error  handling.  The  developmental 
test  environment  was  initially  limited  because  of  no 
operational  data  from  the  legacy  system;  however,  this  has 
now  been  corrected. 

•  The  transition  process  from  EKMS  to  KMI  functioned 
in  DT,  but  was  inadequate  once  implemented  in  the 
operational  environment  on  live  networks.  The  controlled 
test  environment  did  not  account  for  multiple  network 
configuration;  and  therefore,  the  test  team  was  forced  to 
perform  rapid  diagnosis,  on-the-fly  troubleshooting,  and 
resolution  as  the  OT&E  was  underway. 

Mine  Resistant  Ambush  Protected  (MRAP)  Caiman 
Multi-Terrain  Vehicle  (CMTV) 

Another  major  capability  insertion  during  FY12  included  the 
Independent  Suspension  System  for  the  CMTV.  Endurance 
testing  of  the  CMTV  is  ongoing  at  Yuma  Proving  Ground, 
Arizona,  in  all  conditions. 

•  Based  on  performance  during  DT,  the  CMTV  cannot  stop 
following  sustained  operations  in  muddy  terrain.  The  program 
suspended  DT  until  the  program  identifies  and  implements  a 
materiel  solution  to  fix  the  brake  system. 

•  The  CMTV  experienced  problems  associated  with  air 
conditioner,  tire,  and  cab  mount  cracking  failures. 

Mine  Resistant  Ambush  Protected  (MRAP)  Dash  Ambulance 

The  MRAP  program  continues  to  acquire  and  test  enhanced 
capabilities  to  integrate  across  the  MRAP  family  of  vehicles.  In 
FY12,  a  major  capability  insertion  included  the  ambulance  kits 
for  the  Navistar  Dash.  The  Dash  Ambulance  is  not  operationally 
effective  and  not  operationally  suitable  because  of  the 
deficiencies  listed  below: 

•  The  patient  compartment  of  the  vehicle  is  small  and  the 
litter  births  are  not  long  enough  to  safely  accommodate  litter 
patients  taller  than  5  feet  11  inches.  A  unit  equipped  with  the 
Dash  Ambulance  cannot  provide  safe  emergency  medical  care 
and  transport  for  tall  casualties  in  close  proximity  to  enemy 
forces.  This  problem  should  have  been  corrected  prior  to  the 
Limited  User  Test  (LUT). 

•  The  small  interior  of  the  Dash  Ambulance  does  not  provide 
sufficient  space  for  medical  equipment  and  inhibits  the  ability 
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of  the  medic  to  maneuver  within  the  compartment  to  properly 
treat  patients. 

Miniature  Air-Launched  Decoy  (MALD)  and  Jammer  (MALD-J) 

The  MALD  variant  (without  the  jammer)  completed  IOT&E 
in  2011  and  was  found  operationally  effective  for  combat,  but 
not  operationally  suitable  due  to  poor  materiel  reliability.  In 
July  2011,  the  Air  Force  identified  a  fault  with  the  missile’s 
radio  frequency  connector  that  caused  it  to  separate  from  the 
missile  during  long-endurance  carriage  flights.  The  Air  Force 
has  repaired  the  fardt  and  conducted  further  reliability  testing; 
however,  MALD’s  operational  reliability  of  78  percent  remains 
below  the  93  percent  threshold  requirement.  The  Air  Force  began 
IOT&E  for  the  MALD-J  variant  in  August  2012. 

MV-22  Osprey 

The  Navy  conducted  Follow-On  Operational  Test  and  Evaluation 
(FOT&E)  in  June  2012  of  the  latest  Block  C  software  and  six 
other  minor  enhancements. 

•  The  Traffic  Advisory  System  (TAS)  became  saturated 
during  formation  flight,  preventing  the  display  of  potentially 
hazardous  traffic  external  to  the  mission  aircraft.  Intended  to 
warn  pilots  of  impending  collision  with  approaching  aircraft, 
the  TAS  does  not  distinguish  between  approaching  aircraft 
and  aircraft  in  formation.  Additional  development  is  needed 
to  address  operational  test  findings  and  improve  the  utility  of 
TAS  for  the  MV-22  fleet. 

Standard  Missile-6  (SM-6) 

The  Navy  completed  SM-6  Phase  2  IOT&E  in  July  2012. 

Phase  2  was  an  extensive  modeling  and  simulation  effort  that 
examined  SM-6  battlespace  not  covered  in  the  flight  tests 
completed  in  July  2011.  As  discussed  last  year,  there  were  two 
classified  performance  anomalies  in  the  flight  test  portion  of 
the  IOT&E  that  a  more  rigorous  DT&E  should  have  discovered 
earlier. 

•  The  Phase  2  modeling  and  simulation  trials  confirmed  the 
classified  performance  deficiency  observed  in  flight  test.  The 
Navy  is  exploring  corrective  actions;  however,  implementation 
and  testing  of  these  corrective  actions  are  not  scheduled. 

•  The  uplink/downlink  antenna  debris  anomaly  was  discovered 
during  DT  and  carried  forward  to  IOT&E  without  corrective 
action  being  fully  implemented  on  all  missiles;  thus,  there 
were  additional  occurrences  during  IOT&E.  The  Navy 
conducted  high-temperature  wind  tunnel  tests,  which 
examined  if  changes  to  the  antenna  sealant  material  and 
insulation  bonding  manufacturing  process  would  eliminate  the 
debris.  The  trials  recorded  no  anomalies  against  these  fixes; 
however,  the  unexpected  discovery  of  insulation  inter-layer 
delamination  on  three  of  five  wind  tunnel  test  articles  raises 
questions  regarding  the  efficacy  of  the  Navy’s  corrective 
actions. 


•  First  observed  in  DT,  the  Mk  54  Safe-Arm  Device  anomaly 
carried  forward  into  IOT&E  with  additional  occurrences. 
While  initially  viewed  as  anomalous,  there  is  not  enough 
evidence  at  this  time  to  determine  whether  the  Mk  54 
behavior,  as  seen  in  testing,  has  a  connection  to  the  burst 
mode  of  the  SM-6.  However,  the  Phase  2  modeling  and 
simulation  trials  confirmed  that  the  missile  lethality  is 
sensitive  to  the  combination  of  the  burst  mode,  target,  and 
engagement  conditions. 

Virginia  Class  Submarine  Modernized  with  the  APB-09 
Acoustic  Rapid  Commercial  Off-the  Shelf  (COTS)  Insertion 
(A-RCI)  Sonar  System  and  AN/BYG-1  Combat  Control  System 

•  A  series  of  Virginia  class  FOT&E  events  examined 
the  mission  performance  changes  as  a  result  of  the 
modernization  of  the  sonar  and  combat  control  system. 

These  tests  were  combined  with  the  operational  evaluations 
of  the  latest  variants  of  the  A-RCI  Sonar  System,  the 
AN/BYG-1  Combat  Control  System,  and  the  Mk  48 
Advanced  Capability  torpedo.  One  of  the  primary  focus 
areas  of  the  new  combat  control  system  software  was  the 
improvement  of  the  Wide  Aperture  Array’s  processing  and 
displays  for  the  operators. 

•  The  Wide  Aperture  Array  demonstrated  poor  performance 
during  the  OT  period,  and  operators  chose  not  to  use  it  to 
aid  in  completing  their  missions.  The  Navy  investigated  the 
problems  after  the  OT  period  was  complete,  developed  new 
software  fixes,  and  fielded  the  new  software  following  some 
limited  DT.  No  OT  has  been  completed  to  evaluate  the  new 
software  or  the  effects  on  mission  performance. 

•  These  problems  are  recurring  and  likely  a  result  of  the 
Navy’s  time -based  process  for  upgrading  electronics 
systems. 

•  Many  other  systems  on  the  Virginia  class  submarine 
exhibited  the  same  failure  modes  in  FOT&E  as  in  IOT&E. 

Warfighter  Information  Network -Tactical  (WIN-T) 

Increment  2 

In  May  2012,  the  Army  conducted  the  WIN-T  Increment  2 
IOT&E  at  Fort  Bliss,  Texas;  White  Sands  Missile  Range,  New 
Mexico;  Fort  Campbell,  Kentucky;  Fort  Riley,  Kansas;  and  Fort 
Gordon,  Georgia.  DOT&E  assessed  the  WIN-T  Increment  2  as 
supportive  of  voice,  video,  and  data  communications  at-the-halt 
and  on-the-move.  However,  the  network  needs  improvement 
in  the  following  areas: 

•  Reliability 

•  Stability  of  the  terrestrial  Highband  Networking  Waveform 
network  to  support  on-the-move  communications 

•  Performance  of  the  Soldier  Network  Extension 

•  Information  Assurance 
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PROBLEMS  DISCOVERED  DURING  EARLY  TESTING  THAT,  IF  NOT  CORRECTED,  COULD  ADVERSELY  AFFECT  MY  ASSESSMENT 
OF  OPERATIONAL  EFFECTIVENESS,  SUITABILITY,  AND  SURVIVABILITY  DURING  INITIAL  OPERATIONAL  TEST  AND 

EVALUATION  (CONDUCTED  WITHIN  THE  NEXT  TWO  YEARS) 


DISCOVERIES  IN  EARLYTESTING  IN  FY1 2 THAT  SHOULD  BE  CORRECTED  PRIORTO  IOT&E 

Bradley  Engineering  Change  Proposal  (ECP) 

Littoral  Combat  Ship  (LCS)  Increment  2 

F-15E  Radar  Modernization  Program  (RMP) 

Multi-Static  Active  Coherent  (MAC)  System 

Joint  Standoff  Weapon  (JSOW)  C-1 

Patriot  Advanced  Capability-3  (PAC-3) 

Bradley  Engineering  Change  Proposal  (ECP) 

In  September  2012,  the  Army  conducted  two  underbody  blast 
tests  at  the  Aberdeen  Test  Center  on  the  M2  A3  Infantry  Fighting 
Vehicle  with  ECP1  components  to  characterize  the  system’s 
vulnerability. 

•  Severe  vehicle  and  occupant  vulnerabilities  were  observed 
during  early  testing.  If  these  vulnerabilities  are  not  corrected 
the  system  will  likely  be  assessed  as  not  survivable  against 
realistic  underbody  threats. 

F-1 5E  Radar  Modernization  Program  (RMP) 

F-15E  RMP  developmental  flight  testing  began  in  January  2011 
and  IOT&E  was  expected  to  begin  in  late  FY12.  The  planned 
FY12  IOT&E  start  did  not  occur  due  to  challenges  in  maturing 
system  software  to  meet  the  user’s  functional  requirements. 

•  Software  stability  is  crucial  to  operational  effectiveness  and 
suitability.  Flowever,  the  program  experienced  software 
maturation  challenges  and  was  unable  to  complete  DT  in  2012. 
Unanticipated  software  performance  shortfalls  led  to  multiple 
radar  software  releases  and  associated  regression  testing 

to  mature  radar  mode  functionality.  At  the  end  of  FY12, 

RMP  performance  had  not  yet  met  the  user’s  requirements. 
Achieving  the  Air  Force  RMP  software  stability  requirement 
by  IOT&E  may  not  be  feasible. 

Joint  Standoff  Weapon  (JSOW)  C-1 

The  Navy  completed  DT  and  initiated  integrated  testing  of  the 
AGM-154C-1  JSOW  variant  during  FY12.  The  JSOW  C-1 
integrated  testing  completed  in  early  FY13,  with  OT  to  begin  in 
mid-FY13. 

•  JSOW  C-1  reliability  is  well  below  the  threshold  primarily 
because  of  software-driven  problems.  Achieving  an  adequate 
assessment  of  Mean  Flight  Hour  between  Operational  Mission 
Failure  during  OT  is  an  area  of  high  risk. 

•  The  pilot-vehicle  interface  is  excessively  complicated  and 
could  prevent  successful  mission  execution. 

Littoral  Combat  Ship  (LCS) 

The  Navy  conducted  shore-based  testing  of  the  MH-60S  Block  2 
Airborne  Mine  Countermeasures  System,  which  is  intended  to 
support  LCS  mine  countermeasures.  Additionally,  the  Navy 


commenced  a  Quick  Reaction  Assessment  (QRA)  of  the  gun 

systems  on  LCS  1.  Testing  indicated  shortfalls  in  performance: 

•  The  Navy  determined  the  MH-60S  helicopter  cannot  safely 
tow  the  AN/AQS-20A  Sonar  Mine  Detecting  Set  (AQS-20A) 
or  the  Organic  Airborne  Sweep  and  Influence  System  because 
the  helicopter  is  underpowered  for  these  operations.  The 
MH-60S  helicopter  will  no  longer  be  assigned  these  missions 
operating  from  any  ship,  including  LCS. 

•  Preliminary  evaluation  of  test  data  collected  during  the 
operational  assessment  (OA)  of  the  MH-60S  Block  2 
Airborne  Laser  Mine  Detection  System  indicates  that 
the  system  does  not  meet  Navy  requirements  for  False 
Classification  Density  and  has  low  reliability. 

•  Results  from  the  QRA  of  the  LCS  gun  systems  revealed 
performance,  reliability,  and  operator  training  deficiencies  for 
both  the  30  mm  and  57  mm  guns. 

Multi-Static  Active  Coherent  (MAC)  System 

The  Navy  conducted  DT  in  2012  and  plans  to  begin  OT  in  early 

FY13. 

•  No  significant  problems  have  been  observed  in  DT  to  date; 
however,  little  realistic  DT  has  been  conducted,  and  the  test 
construct  used  for  DT  contained  target  requirements  that  may 
support  model  verification  but  were  not  operationally  realistic 
or  translatable  to  operationally  realistic  conditions.  The  Navy 
plans  to  waive  two  known  problems  that  will  likely  affect 
mission  performance. 

Patriot  Advanced  Capability-3  (PAC-3) 

The  Army  completed  DT  of  the  Post-Deployment  Build-7 

(PDB-7)  and  began  a  LUT  operational  test  in  FY12. 

•  Data  analysis  is  ongoing,  but  preliminary  results  indicate 
that  Patriot  training  remains  inadequate  to  prepare  operators 
for  complex  Patriot  engagements.  This  was  true  during 

the  PDB-6.5  and  PDB-6  LUTs  as  well.  This  problem 
was  exacerbated  in  the  PDB-7  LUT  because  many  of 
the  experienced  Patriot  operators  in  the  test  unit  were 
understandably  transferred  to  deploying  units  prior  to  the 
LUT,  resulting  in  many  inexperienced  users  and  a  high 
variability  in  Soldier  proficiency  across  the  test  unit. 
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The  Patriot  system  did  not  meet  its  reliability  requirements 
during  the  PDB-7  DT.  DOT&E  is  investigating  the  possibility 
of  using  field  data  to  improve  the  estimates  of  Patriot  system 
reliability  such  as  Mean  Time  Between  Critical  Mission 


Failure.  However,  critical  field  data  including  total  operating 
hours  and  numbers  of  critical  mission  failures  for  each  Patriot 
battery  major  end  item  may  not  be  accurate. 


CONCLUSION 


Previously,  Congress  has  expressed  concerns  that  significant 
weapons  acquisition  program  problems  are  discovered  during 
OT&E  that  should  have  been  discovered  during  DT.  Last  year, 
I  documented  40  systems  with  significant  discovery  during 
OT  during  2010-2011;  23  of  those  systems  had  discovery 
in  early  OT,  of  which  19  implemented  fixes  that  were  either 
verified  by  successful  IOT&E  or  are  currently  in  IOT&E.  Of 
the  1 7  programs  that  discovered  significant  issues  during  their 


IOT&E  in  2010-2011,  12  have  implemented  fixes  that  were  either 
verified  in  successful  FOT&E  or  are  planning  additional  OT 
periods;  2  of  the  remaining  5  programs  were  cancelled.  Thus, 
while  significant  issues  are  being  discovered  late  in  the  programs’ 
acquisition  cycle,  most  programs  are  addressing  the  discoveries 
and  verifying  fixes  in  FOT&E.  In  2012,  17  programs  had 
significant  discoveries  in  IOT&E  or  FOT&E,  while  7  programs 
had  significant  discovery  in  early  testing. 
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EProcurement 


Executive  Summary 

•  The  Joint  Interoperability  Test  Command  (JITC)  conducted  a 
DOT&E-directed  Operational  Assessment  (OA)  of  Release  1.2 
in  January  2012.  EProcurement  demonstrated  significantly 
improved  maturity  over  the  previous  Release  1.1.  Users 
could  successfully  employ  the  system  to  execute  all  phases  of 
procurement  mission  tasks  with  few  problems.  However,  user 
perception  of  training  adequacy,  system  usability,  and  system 
timeliness  remained  poor. 

•  JITC  conducted  the  IOT&E  of  Release  1.2  between 
February  and  April  2012. 

•  EProcurement  is  operationally  effective.  The  effectiveness 
evaluation  concentrated  on  the  users’  ability  to  use 
EProcurement  in  six  areas:  manage  purchase  requisitions, 
source  and  solicit  goods  and  services,  manage  awards, 
process  receipts  and  invoices,  create  reports,  and  maintain 
system  data  (data  cleansing  and  conversions).  JITC  observed 
users  performing  day-to-day  operations  and  recorded  1,350 
successful  transactions  and  13  failed  transactions  (99  percent 
success  rate). 

•  EProcurement  is  operationally  suitable,  but  with  deficiencies. 
Improvements  are  required  in  the  areas  of  training,  usability, 
Help  Desk  operations,  and  supportability. 

•  EProcurement  is  secure  from  an  Information  Assurance 
perspective.  At  the  end  of  IOT&E,  only  one  moderate 
impact,  and  two  low  impact  deficiencies  remained  open, 
with  minimal  effect  on  system  security  and  operations. 

The  Defense  Information  Systems  Agency  (DISA)  and  the 
Defense  Logistics  Agency  (DLA)  created  a  plan  of  action 
and  milestones  to  address  resolution  of  these  deficiencies. 
However,  DOT&E  was  unable  to  verify  that  DLA  has  a  robust 
theft  and  fraud  prevention  and  detection  program. 

•  The  Deputy  Chief  Management  Officer  (DCMO)  approved 
the  Full  Deployment  Decision  (FDD)  of  EProcurement  on 
August  23,  2012.  As  a  part  of  the  FDD  approval,  DLA 
agreed  to  employ  a  Red  Team  to  conduct  a  pilot  assessment 
in  accordance  with  the  rules  of  engagement  established  by  the 
results  of  the  DCMO-sponsored  Massachusetts  Institution  of 
Technology  Theft/Fraud  Red  Team  study.  DLA  also  agreed 
to  continue  the  effort  to  demonstrate  automated  regression 
testing  as  a  potential  standard  model  for  the  DoD. 


Folder,  Procurement  Automated  Contract  Evaluation,  and 
Base  Operations  Support  System). 

•  The  Intended  functions  of  EProcurement  include:  manage 
purchase  requisition,  source  and  solicit,  manage  award, 
manage  vendor  performance,  perform  receipt,  and  process 
invoice.  Receipts  and  invoices  are  processed  via  the  DLA 
Wide  Area  Workflow  System  or  manually. 

•  EProcurement  is  based  on  a  suite  of  commercial  off-the-shelf 
(COTS)  software  applications  purchased  from  the  Systems 
Applications  and  Products  Corporation  that  operates  on  COTS 
computing  hardware. 

•  The  Defense  Enterprise  Computing  Center  (DECC)  in  Ogden, 
Utah,  which  is  operated  and  maintained  by  DISA,  is  the 
production  environment  for  EProcurement.  EProcurement  is 
one  of  the  programs  in  the  overall  DLA  Enterprise  Business 
System  Infrastructure  hosted  by  the  Ogden  DECC.  The 
back-up  site  is  located  at  the  DECC  in  Mechanic sburg, 
Pennsylvania. 

Mission 

DLA  users  will  use  EProcurement  to  procure  and  provide  the  full 
spectrum  of  consumables,  services,  and  depot-level  repairables  to 
the  Army,  Navy,  Air  Force,  Marine  Corps,  other  federal  agencies, 
and  combined  and  allied  forces. 


System  Major  Contractor 

•  DLA  developed  EProcurement  to  provide  enterprise-level  Accenture  -  Reston,  Virginia 

procurement  capabilities  to  replace  legacy  procurement 
systems  (Pre-Award  Contracting  System,  Electronic  Contract 


EProcurement 


23 


DOD  PROGRAMS 


Activity 

•  JITC  conducted  an  OA  of  Release  1 .2  in  January  2012, 
in  accordance  with  a  DOT&E-approved  OA  plan.  JITC 
conducted  the  OA  at  the  DLA  Aviation  facility  in  Richmond, 
Virginia. 

•  JITC  conducted  the  IOT&E  of  EProcurement  Release  1.2 
from  February  through  April  2012.  JITC  executed  the 
test  at  DLA  locations  in  New  Cumberland,  Pennsylvania; 
Mechanicsburg,  Pennsylvania;  Battle  Creek,  Michigan;  Fort 
Belvoir,  Virginia;  and  Richmond,  Virginia,  in  accordance  with 
a  DOT&E-approved  IOT&E  plan. 

•  Information  Assurance  testers  from  JITC  along  with  members 
of  the  DISA  Field  Security  Operations  (FSO)  and  DLA 
Computer  Emergency  Response  Team  (CERT)  conducted 
penetration  test  events  at  the  DECC  in  Ogden,  Utah,  during 
both  the  OA  and  IOT&E. 

Assessment 

•  During  the  Release  1 .2  OA,  EProcurement  demonstrated 
significantly  improved  maturity  over  the  previous  Release  1.1. 
Users  successfully  used  the  system  to  execute  all  phases  of 
procurement  mission  tasks  with  few  problems.  However,  user 
training,  system  usability,  and  system  timeliness  still  require 
improvement. 

•  Based  on  the  IOT&E  results,  EProcurement  is  operationally 
effective. 

-  The  effectiveness  evaluation  concentrated  on  the  users’ 
ability  to  use  EProcurement  to:  manage  purchase 
requisitions,  source  and  solicit  goods  and  services,  manage 
awards,  process  receipts  and  invoices,  create  reports,  and 
maintain  system  data  (data  cleansing  and  conversions), 
which  included  all  major  system  functionality. 

-  JITC  observed  users  performing  day-to-day  operations 
and  recorded  1,350  successful  transactions  and  13  failed 
transactions,  which  achieved  a  99  percent  success  rate  and 
exceeded  the  90  percent  requirement. 

-  EProcurement  is  interoperable.  JITC  interoperability 
testers  evaluated  52  interfaces  to  assess  the  data  exchanges 
between  EProcurement  and  other  systems,  with  no  failures 
reported  in  the  more  than  400  transactions  evaluated. 

•  EProcurement  is  operationally  suitable  but  has  deficiencies 
in  the  areas  of  training,  usability,  Help  Desk  operations,  and 
supportability. 

-  DOT&E  considers  EProcurement  reliable,  available,  and 
maintainable.  DLA  reported  one  site  specific  outage  at 
Battle  Creek,  Michigan,  lasting  1  hour  and  55  minutes,  but 
it  appears  to  have  been  an  isolated  event. 

-  EProcurement  training,  training  aids,  and  system 
documentation  need  improvement.  None  of  these  areas 
met  the  80  percent  threshold  of  acceptability  by  the  users 
surveyed.  Users  report  that  EProcurement  is  not  user 
friendly  and  is  difficult  to  master. 

-  Help  Desk  trouble  tickets  take  a  long  time  to  resolve. 
Analysis  of  the  closure  rates  for  the  tickets  shows  that 
the  average  time  required  to  resolve  trouble  tickets  was 


approximately  8.5  days,  with  resolution  times  ranging  from 
less  than  5  minutes  to  nearly  40  days. 

-  The  supportability  of  EProcurement  needs  improvement  as 
DLA  does  not  have  an  automated  test  capability  to  perform 
thorough  regression  testing  on  new  software  releases. 

•  EProcurement  meets  Information  Assurance  security 
thresholds.  JITC  Information  Assurance  testers,  along  with 
members  of  the  DISA  FSO  and  DLA  CERT,  conducted  a 
series  of  penetration  test  events  primarily  at  the  Ogden  DECC. 

-  Following  IOT&E,  only  one  moderate  impact,  and  two  low 
impact  deficiencies  remained  open,  with  minimal  effect  on 
system  security  and  operations.  DISA  and  DLA  created  a 
plan  of  action  and  milestones  to  address  resolution  of  these 
deficiencies. 

-  DLA  did  not  implement  a  robust  theft  and  fraud  prevention 
and  detection  program.  DLA  asserts  that  by  using  role 
separations,  at  least  two  to  three  people  would  need  to 

be  involved  in  any  theft  or  fraud  activity  and  that  while 
large-scale  theft  or  fraud  was  not  impossible,  it  would  be 
difficult.  DLA  also  indicated  that  bi-annual  audits  and 
other  accounting  controls  are  in  place  to  further  mitigate 
such  activities.  While  DOT&E  acknowledges  that  some 
level  of  prevention  and  detection  is  available  at  DLA, 
the  extent  of  the  financial  threat  vulnerability  is  yet  to  be 
determined  and  is  pending  the  outcome  of  a  financial  theft 
and  fraud  test. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program 
manager  satisfactorily  addressed  three  of  the  five  FY 1 1 
recommendations.  The  program  manager  partially  addressed 
recommendations  concerning  operational  realism  of 
developmental  testing  and  the  use  of  automated  test  tools; 
these  recommendations  remain  valid. 

•  FY  12  Recommendations. 

1.  DLA  should  employ  a  Theft  and  Fraud  Prevention  and 
Detection  Red  Team  to  conduct  a  pilot  assessment  in 
accordance  with  the  rides  of  engagement  established  by  the 
results  of  the  Deputy  Chief  Management  Officer-sponsored 
Theft  and  Fraud  Red  Team  study. 

2.  DLA  should  continue  with  ongoing  efforts  to  implement 
automated  regression  testing. 

3.  DLA  should  improve  the  quality  of  training,  training  aids, 
and  other  system  documentation  for  the  users,  and  include 
role-specific  training  in  the  future  when  DLA  transitions 
users  to  EProcurement  at  the  remaining  DLA  sites. 

4.  DLA  should  modify  the  method  of  managing  trouble  tickets 
to  allow  for  better  query  capabilities.  DLA  should  also 
periodically  track  the  resolution  times  of  system  problems 
to  aid  DLA  management  in  identifying  potential  problem 
areas  so  that  DLA  can  implement  mitigation  strategies 
before  productivity  is  affected. 

5.  DLA  should  periodically  administer  the  System  Usability 
Scale  survey  to  a  random  sample  of  all  EProcurement  users 
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through  full  deployment  to  see  whether  user  satisfaction 
does  improve  with  increased  system  use  and  to  pinpoint 
any  inherent  deficiencies  with  system  usability. 

6.  In  future  testing,  J1TC  and  DLA  should  evaluate  all 

untested  interfaces  that  will  be  part  of  the  full  deployment. 


7.  Although  only  minor  deficiencies  remained  after  the  last 
Information  Assurance  test  event,  the  DISA  FSO  and  DLA 
CERT  should  periodically  reevaluate  the  security  posture  of 
the  system  as  part  of  the  overall  defense-in-depth  security 
strategy. 
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F-35  Joint  Strike  Fighter  (JSF) 


Executive  Summary 

•  The  F-35  Joint  Strike  Fighter  (JSF)  program  continues 
to  have  a  high  level  of  concurrency  among  production, 
development,  and  test.  Approximately  34  percent  of  the  total 
planned  flight  testing,  based  on  test  points  completed  through 
November  2012,  has  now  been  accomplished  as  the  program 
initiates  the  fifth  of  11  initial  production  lots.  Durability 
testing  is  ongoing  on  all  three  variants,  with  only  the  F-35A 
test  article  having  completed  a  full  lifetime  of  testing.  The 
program  will  not  complete  the  two  lifetimes  of  durability 
testing  currently  planned  on  any  variant  until  the  last  quarter 
of  2014. 

•  Through  November  2012,  the  flight  test  teams  were  able  to 
exceed  the  flight  rate  planned  for  flight  sciences  in  the  F-35B 
and  F-35C  variants,  but  were  slightly  behind  the  plan  for  the 
F-35  A.  The  program  did  not  accomplish  the  intended  progress 
in  achieving  test  objectives  (measured  in  flight  test  points 
planned  for  2012)  for  all  variants.  Certain  test  conditions  were 
unachievable  due  to  unresolved  problems  and  new  discoveries. 
The  need  for  regression  testing  of  fixes  (repeat  testing  of 
previously  accomplished  points  with  newer  versions  of 
software)  displaced  opportunities  to  meet  flight  test  objectives. 

•  The  flight  rate  of  the  mission  systems  test  aircraft  also 
exceeded  the  planned  rate  during  the  year,  but  overall  progress 
in  mission  systems  was  limited.  This  was  due  to  delays  in 
software  delivery,  limited  capability  in  the  software  when 
delivered,  and  regression  testing  of  multiple  software  versions 
(required  to  fix  problems,  not  add  capability).  Test  points 
accomplished  for  the  year  included  Block  1  verification, 
validation  of  limited  capabilities  for  early  lot  production 
aircraft,  baseline  signature  testing,  and  Block  2  development. 
No  combat  capability  has  been  fielded. 

•  The  lag  in  accomplishing  the  intended  2012  flight  testing 
content  defers  testing  to  following  years,  and  in  the  meantime, 
will  contribute  to  the  program  delivering  less  capability  in 
production  aircraft  in  the  near  term. 

•  The  tables  on  the  following  page  present  the  actual  versus 
planned  test  flights  and  test  points  conducted  as  of  the  end  of 
November  2012. 

•  The  program  submitted  Revision  4  of  the  Test  and  Evaluation 
Master  Plan  (TEMP)  for  approval,  which  included  changes 
to  the  program  structure  brought  about  by  the  previous  year’s 
Technical  Baseline  Review  and  subsequent  re-planning  of 
testing.  Flowever,  the  TEMP  contained  an  unacceptable 
overlap  of  development  with  the  start  of  operational  test 
activity  for  IOT&E. 

•  The  Air  Force  began  the  F-35A  training  Operational  Utility 
Evaluation  (OUE)  in  September  2012  and  completed  it  in 
mid-November.  During  the  OUE,  four  pilots  completed 
training  in  the  system  familiarization  portion  of  the 
syllabus,  which  included  no  combat  capabilities.  Because 


of  the  immaturity  of  the  system,  which  is  still  largely 
under  development,  little  can  be  learned  about  operating 
and  sustaining  the  F-35  in  combat  operations  from  this 
evaluation. 

•  The  program  completed  two  of  the  eight  planned 
system-level  ballistic  test  series. 

-  The  first  series  confirmed  the  built-in  redundancies  and 
reconfiguration  capabilities  of  the  flight-critical  systems. 
The  second  series  indicated  that  ballistic  damage 
introduced  no  measurable  degradation  in  the  F-35B 
propulsion  system  performance  and  that  the  damage 
would  be  undetectable  by  the  pilot.  Ongoing  analysis  will 
evaluate  whether  these  tests  stressed  the  vulnerabilities 
unique  to  ballistic  damage  to  the  F-35  (e.g.,  interference 
or  arcing  between  270  Volt,  28  Volt,  and  signal  lines 
and/or  damage  to  lift  fan  blade  sections). 

-  The  first  test  series  confirmed  Polyalphaolefin  (PAO) 
coolant  and  fueldraulic  systems  fire  vulnerabilities.  The 
relevant  protective  systems  were  removed  from  the 
aircraft  in  2008  as  part  of  a  weight  reduction  effort.  A 
Computation  of  Vulnerable  Area  Tool  analysis  shows 
that  the  removal  of  these  systems  results  in  a  25  percent 
increase  in  aircraft  vulnerability.  The  F-35  Program 
Office  may  consider  reinstalling  the  PAO  shutoff  valve 
feature  based  on  a  more  detailed  cost-benefit  assessment. 
Fueldraulic  system  protection  is  not  being  reconsidered 
for  the  F-35  design. 

•  The  program’s  most  recent  vulnerability  assessment  showed 
that  the  removal  of  fueldraulic  fuses,  the  PAO  shutoff  valve, 


F-35  JSF  27 


DOD  PROGRAMS 


and  the  dry  bay  fire  suppression,  also  removed  in  2008, 
results  in  the  F-35  not  meeting  the  Operational  Requirements 
Document  (ORD)  requirement  to  have  a  vulnerability  posture 
better  than  analogous  legacy  aircraft. 

•  Tests  of  the  fuel  tank  inerting  system  in  2009  identified 

deficiencies  in  maintaining  the  required  lower  fuel  tank  oxygen 


levels  to  prevent  fuel  tank  explosions.  The  system  is  not  able 
to  maintain  fuel  tank  inerting  through  some  critical  portions  of 
a  simulated  mission  profile.  The  program  is  redesigning  the 
On-Board  Inert  Gas  Generating  System  (OBIGGS)  to  provide 
the  required  levels  of  protection  from  threat  and  from  fuel  tank 
explosions  induced  by  lightning. 


Actual  versus  Planned  Test  Metrics  through  November  2012 


TEST  FLIGHTS 

All  Testing 

Flight  Sciences 

Mission 

Systems 

All  Variants 

F-35B  Only 

F-35A  Only 

F-35C  Only 

201 2  Actual 

1,092 

374 

263 

233 

222 

2012  Planned 

927 

244 

279 

211 

193 

Difference  from  Planned 

+18% 

+53% 

-6% 

+10% 

+15% 

Cumulative  Actual 

2,533 

963 

709 

425 

436 

Cumulative  Planned 

2,238 

820 

651 

404 

363 

Difference  from  Planned 

+13% 

+17% 

+9% 

+5% 

+20% 

TEST  POINTS 


All  Testing 

Flight  Sciences 

Mission  Systems 

All  Variants 

F-35B  Only 

F-35A  Only 

F-35C  Only 

Block  1* 

Block  2 

Block  3 

Other 

201 2  Baseline  Accomplished 

4,711 

1,075 

1,338 

1,060 

358 

457 

0 

423 

2012  Baseline  Planned 

6,497 

1,939 

1,923 

1,327 

336 

448 

0 

524 

Difference  from  Planned 

-28% 

-45% 

-30% 

-20% 

+7% 

+2% 

0 

-19% 

Added  Points 

1,720 

292 

565 

253 

0 

610 

0 

0 

Points  from  Future  Year  Plans 

2,319 

992 

431 

896 

0 

0 

0 

0 

Total  Points  Accomplished** 

8,750 

2,359 

2,334 

2,209 

358 

1,067 

0 

423 

Cumulative  SDD  Actual*** 

20,006 

7,480 

5,664 

4,330 

899 

457 

0 

1,176 

Cumulative  SDD  Planned 

19,134 

7,057 

6,102 

3,748 

667 

488 

0 

1,072 

Difference  from  Planned 

+5% 

+6% 

-7% 

+16% 

+35% 

-6% 

+10% 

Test  Points  Remaining 

39,579 

12,508 

8,321 

10,316 

8,434  (All  Blocks  and  Other  Mission  Systems  Activity) 

*  Includes  Block  0.5  and  Block  1  quantities 
**  Total  Points  Accomplished  =  2012  Baseline  Accomplished  +  Added  Points  +  Points  from  Future  Year  Plans 

***  SDD  -  System  Design  and  Development 


System 

•  The  F-35  JSF  program  is  a  tri-Service,  multi-national, 
single-seat,  single-engine  family  of  strike  aircraft  consisting  of 
three  variants: 

-  F-35A  Conventional  Take-Off  and  Landing  (CTOL) 

-  F-35B  Short  Take-Off/Vertical-Landing  (STOVL) 

-  F-35C  Aircraft  Carrier  Variant  (CV) 

•  It  is  designed  to  survive  in  an  advanced  threat  (year  2012  and 
beyond)  environment  using  numerous  advanced  capabilities. 

It  is  also  designed  to  have  improved  lethality  in  this 
environment  compared  to  legacy  multi-role  aircraft. 

•  Using  an  Active  Electronically  Scanned  Array  (AESA) 
radar  and  other  sensors,  the  F-35  is  intended  to  employ 
precision-guided  bombs  such  as  the  Joint  Direct  Attack 
Munition  (JDAM)  and  Joint  Standoff  Weapon,  AIM-120C 
radar-guided  Advanced  Medium-Range  Air-to-Air  Missile 
(AMRAAM),  and  AIM-9  infrared-guided  short-range  air-to-air 
missile. 


•  The  program  provides  mission  capability  in  three  increments: 
Block  1  (initial  training).  Block  2  (advanced),  and  Block  3  (full). 

•  The  F-35  is  under  development  by  a  partnership  of  countries: 
the  United  States,  Great  Britain,  Italy,  the  Netherlands,  Turkey, 
Canada,  Australia,  Denmark,  and  Norway. 

Mission 

•  A  force  equipped  with  F-35  units  should  permit  the  Combatant 
Commander  to  attack  targets  day  or  night,  in  all  weather,  and 
in  highly  defended  areas  of  joint  operations. 

•  F-35  will  be  used  to  attack  fixed  and  mobile  land  targets, 
enemy  surface  units  at-sea,  and  air  threats,  including  advanced 
cruise  missiles. 

Major  Contractor 

Lockheed  Martin,  Aeronautics  Division  -  Fort  Worth,  Texas 
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Test  Strategy,  Planning,  and  Resourcing 

•  The  JSF  Program  Office,  in  coordination  with  the 
operational  test  agencies,  worked  to  develop  Revision  4 
of  the  TEMP.  As  part  of  the  Milestone  B  recertification 
in  March  2012,  the  USD(AT&L)  tasked  the  program  to 
submit  a  revised  TEMP  for  approval  prior  to  the  September 
In-Progress  Review  by  the  Defense  Acquisition  Board. 

•  The  TEMP  included  a  schedule  for  IOT&E  that  assumed 
the  final  preparation  period  prior  to  IOT&E  could  fully 
overlap  with  the  air-worthiness  certification  phase  of 
development,  which  occurs  after  the  final  developmental 
test  events.  DOT&E  identified  to  the  program  and  the 
JSF  Operational  Test  Team  that  without  analysis  showing 
this  overlap  is  feasible,  the  TEMP  could  not  be  approved. 
DOT&E  concluded  that  this  final  preparation  period  should 
be  scheduled  to  begin  at  a  later  point,  no  earlier  than  the 
Operational  Test  Readiness  Review,  and  budgets  should  be 
adjusted  accordingly. 

•  This  report  reviews  the  program  by  analyzing  the  progress 
of  testing  and  the  capability  delivered  as  a  function  of  test 
results.  The  program  plans  a  specific  set  of  test  points 
(discrete  measurements  of  performance  under  specific  test 
conditions)  for  accomplishment  in  a  given  calendar  year.  In 
this  report,  test  points  planned  for  a  given  calendar  year  are 
referred  to  as  baseline  test  points.  In  addition  to  baseline 
test  points,  the  program  accomplishes  test  points  added 

for  discovery  and  regression.  Cumulative  System  Design 
and  Development  (SDD)  test  point  data  refer  to  the  total 
progress  towards  completing  development  at  the  end  of 
SDD. 

F-35A  Flight  Sciences 

Flight  Test  Activity  with  AF-1,  AF-2,  andAF-4  Test  Aircraft 

•  F-35A  flight  sciences  testing  focused  on: 

-  Expanding  the  flight  envelope  (achieved  700  knots 
calibrated  airspeed  [KCAS]/1.6  Mach  test  point  in 
March  and  achieved  50,000  feet,  the  designed  altitude 
limit,  in  November) 

-  Evaluating  flying  qualities  with  internal  stores  (GBU-3 1 
JDAM,  GBU-12  Laser-guided  Bomb,  and  AIM-120 
AMRAAM)  and  external  stores  (AIM-9X  short-range 
missile) 

-  Characterizing  subsonic  and  supersonic  weapons  bay 
door  and  environment 

-  Expanding  the  air-refueling  envelope  and  investigating 
tanker-to-F-35A  connection/disconnection  problems 

-  Engine  air-start  testing 

-  High  (greater  than  20  degrees)  angle-of-attack  testing 

•  The  test  team  began  weapons  separation  testing  in  October 
with  the  first  safe  separation  of  an  inert  GBU-3 1  JDAM, 
followed  by  the  first  AIM- 120  safe  separation  later  in  the 
month. 

•  The  program  released  two  revisions  of  the  air  vehicle 
systems  software  (R27.1  and  R27.2.2)  in  2012  to  improve 


flying  qualities,  correct  air  data  deficiencies  observed 
during  F-35A  envelope  expansion,  and  to  address  various 
software  deficiencies. 

•  Through  the  end  of  November  2012,  the  test  team  was  able 
to  sustain  a  sortie  rate  of  8.0  flights  per  aircraft  per  month, 
compared  to  the  goal  of  8.5  sorties  per  month.  The  overall 
annual  sortie  total  was  only  6  percent  short  of  the  goal 
(263  sorties  completed,  279  planned). 

Flight  Sciences  Assessment 

•  By  the  end  of  November,  the  progress  against  planned 
baseline  test  points  for  2012  lagged  by  over  30  percent 
(accomplishing  1,338  baseline  F-35Aflight  sciences  test 
points  of  1,923  planned  through  November  2012,  for  a 
completion  rate  of  70  percent).  The  test  team  could  not 
execute  this  portion  (30  percent)  of  planned  2012  baseline 
test  points  for  the  following  reasons: 

-  Aircraft  operating  limitations,  which  prevented  the 
extended  use  of  afterburner  needed  to  complete 
high-altitude/high-airspeed  test  points. 

-  Higher  than  expected  loads  on  the  weapon  bay  doors, 
which  required  additional  testing  and  thus  limited  the 
amount  of  testing  with  weapons  loaded  on  the  aircraft. 

-  Deficiencies  in  the  air-refueling  system,  which  reduced 
testing  opportunities. 

•  To  compensate  for  not  being  able  to  achieve  the  baseline 
test  points  planned  for  2012,  the  test  team  moved  up  test 
points  planned  for  completion  in  later  years,  and  was 
thereby  able  to  nearly  keep  pace  with  overall  cumulative 
SDD  test  point  objectives.  For  example,  the  Block  2B 
flight  envelope  includes  operations  with  the  weapons  bay 
doors  open.  The  program  discovered  dynamic  flight  loads 
on  portions  of  the  open  doors  were  higher  than  expected, 
requiring  additional  instrumentation  and  testing.  The  test 
team  substituted  other  test  points,  which  were  available 
from  Block  3  envelope  plans  for  2013  that  did  not  require 
the  doors  open.  For  F-35A  flight  sciences,  the  test  team 
had  accomplished  93  percent  of  the  overall  planned  number 
of  cumulative  test  points  scheduled  for  completion  by  the 
end  of  November  (5,664  cumulative  points  accomplished 
against  a  goal  of  6,102  points). 

•  Weight  management  of  the  F-35A  variant  is  important 
for  meeting  air  vehicle  performance  requirements.  The 
program  generates  monthly  aircraft  weight  status  reports 
for  all  variants  and  computes  weights  as  a  sum  of  measured 
weights  of  components  or  subassemblies,  calculated 
weights  from  approved  design  drawings  released  for  build, 
and  engineering  weight  estimates  of  remaining  components. 
The  program  has  managed  to  keep  F-35A  weight  estimates 
nearly  constant  for  the  last  year.  The  latest  F-35A  weight 
status  report  from  November  2012  showed  the  estimated 
weight  of  29,098  pounds  to  be  within  273  pounds 

(0.94  percent)  of  the  projected  maximum  weight  needed 
to  meet  the  technical  performance  required  per  contract 
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specifications  in  January  2015.  This  small  margin  allows 
for  only  0.42  percent  weight  growth  per  year  for  the 
F-35A.  The  program  will  need  to  continue  rigorous  weight 
management  through  the  end  of  SDD  to  avoid  performance 
degradation  and  operational  impacts. 

•  The  program  announced  an  intention  to  change 
performance  specifications  for  the  F-35A,  reducing  turn 
performance  from  5.3  to  4.6  sustained  g’s  and  extending 
the  time  for  acceleration  from  0.8  Mach  to  1.2  Mach  by 
8  seconds.  These  changes  were  due  to  the  results  of  air 
vehicle  performance  and  flying  qualities  evaluations. 

•  Discoveries  included: 

-  Delayed  disconnects  during  air  refueling  required  the 
program  to  implement  restrictions  on  the  F-35A  fleet 
and  conduct  additional  testing  of  the  air  refueling 
capability.  The  program  added  instrumentation  to 
isolate  root  causes. 

-  Horizontal  tail  surfaces  are  experiencing  higher 
than  expected  temperatures  during  sustained 
high-speed/high-altitude  flight,  resulting  in 
delamination  and  scorching  of  the  surface  coatings 
and  structure.  All  variants  were  restricted  from 
operations  outside  of  a  reduced  envelope  until  the 
test  team  added  instrumentation  to  the  tailbooms  to 
monitor  temperatures  on  the  tail  surfaces.  The  program 
scheduled  modification  of  one  flight  sciences  aircraft  of 
each  variant  with  new  skin  coatings  on  the  horizontal 
tail  to  permit  flight  testing  in  the  currently  restricted  part 
of  the  high-speed/high-altitude  flight  envelope.  The 
test  team  is  adding  more  flight  test  instrumentation  to 
help  quantify  the  impacts  of  the  tail  heating  to  support 
necessary  design  changes.  The  program  scheduled 
modifications  on  one  aircraft  (AF-2)  to  be  completed  in 
early  2013  to  allow  flight  testing  of  the  new  skin  design 
on  the  horizontal  tails  to  proceed. 

F-35B  Flight  Sciences 

Flight  Test  Activity  with  BF-1,  BF-2,  BF-3,  BF-4,  and  BF-5 

Test  Aircraft 

•  F-35B  flight  sciences  focused  on: 

-  Expansion  of  the  vertical-lift  operations  envelope  testing 
of  the  newly  designed  auxiliary  air  inlet  door 

-  Engine  air-start  testing 

-  Expansion  of  the  flight  envelope  with  weapons  loaded 
on  the  aircraft 

-  Fuel  dump  operations 

-  Regression  testing  of  new  vehicle  systems  software 

•  The  test  team  accomplished  radar  signature  testing  on  BF-5 
after  the  aircraft  was  returned  to  the  plant  for  four  months 
for  final  finishes. 


•  The  test  team  began  weapon-separation  flight  tests  in 
August  when  BF-5  accomplished  a  successful  safe 
separation  of  an  inert  GBU-32  JDAM. 

•  As  of  the  end  of  November,  the  sortie  rate  for  the  F-35B 
flight  sciences  test  aircraft  was  6.8  sorties  per  aircraft 
per  month,  compared  to  the  goal  of  4.4.  The  program 
accomplished  153  percent  of  the  planned  F-35B  flight 
sciences  sorties,  completing  374  vice  244  planned. 

Flight  Sciences  Assessment 

•  Although  the  program  exceeded  the  objectives  planned 
for  sortie  rate  through  the  end  of  November,  the  progress 
against  planned  baseline  test  points  for  2012  lagged  by 
45  percent  with  1,075  test  points  accomplished  against 
1,939  planned.  This  was  primarily  a  result  of  higher-than- 
expected  loads  on  weapon  bay  doors,  which  prevented 
planned  envelope  expansion  test  points  and  required 
additional  unplanned  testing. 

•  To  compensate  for  not  being  able  to  accomplish  the 
planned  envelope  expansion  test  points,  the  test  team 
pulled  an  additional  992  points  from  testing  planned  for 
2013  back  into  2012  and  added  292  points  for  regression 
testing  of  new  software.  As  of  the  end  of  November,  the 
program  had  accomplished  2,359  total  test  points  for  the 
year.  By  pulling  test  points  to  2012  that  were  originally 
planned  for  execution  in  later  years,  the  test  team  was 
able  to  keep  pace  with  the  program’s  overall  cumulative 
SDD  test  point  objectives.  Like  the  F-35A,  loads  on  the 
weapons  bay  doors  prevented  test  point  accomplishment 
for  internally-loaded  weapons;  test  points  with  external 
stores  were  accomplished  instead.  For  F-35B  flight 
sciences,  the  test  team  had  completed  106  percent  of  the 
planned  quantity  of  cumulative  SDD  test  points  scheduled 
for  completion  by  the  end  of  November  (7,480  cumulative 
points  accomplished  against  a  goal  of  7,057  points). 

•  The  test  team  continued  investigations  into  the  impact  of 
transonic  roll-off  and  transonic  buffet  in  the  F-35B;  these 
investigations  are  not  complete.  The  program  introduced 
new  F-35B  vehicle  systems  software  to  reduce  rudder  and 
flaperon  hinge  moment  in  the  transonic/supersonic  region. 
The  program  expected  to  see  improvements  in  transonic 
wing  roll-off  with  these  changes,  but  results  were  not 
available  at  the  end  of  November  2012. 

•  The  following  table,  first  displayed  in  the  FY11  Annual 
Report,  describes  the  observed  door  and  propulsion 
problems  by  component  and  identifies  the  production 
cut-in,  if  known.  A  significant  amount  of  flight  test  and 
validation  of  an  adequate  final  STOVL-mode  configuration 
(doors  and  propulsion  system)  remains  to  be  accomplished. 
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F-35  B  Door  and  Propulsion  Problems 


Category 

Component 

Problem 

Design  Fix  and  Test  Status 

Production  Cut-In 

Subsystems 

Upper  Lift  Fan  Inlet 
Door  Actuators 

High  actuator  failure  rates. 

Root  cause  analysis  is  complete  and  failure  modes  are 
limited  to  open  position  (i.e.,  failure  to  close);  the  doors  have 
not  failed  to  open  when  commanded,  which  allows  lift  fan 
operations.  New  actuator  design  is  complete  and  testing  is 
entering  final  stage  of  qualification. 

BF-38 

Low-Rate  Initial 
Production  (LRIP)  6 
2014 

Structure 

Auxiliary  Air  Inlet 
Door  (AAID) 

Inadequate  life  on  door  locks,  excessive 
wear  and  fatigue  due  to  the  buffet 
environment,  inadequate  seal  design. 

Redesigned  door  undergoing  testing  on  BF-1 .  Loads  testing 
completed  and  verified.  Static  testing  on  ground  test  article 
(BG-1 )  is  complete,  fatigue  testing  started  in  November. 

BF-38 

LRIP  6 

2014 

Structure 

Lift  Fan  Door 
Actuator  Support 
Beam 

Cracks  occurring  earlier  than  predicted. 

Root  cause  analysis  showed  fastener 
location  incorrectly  inserted  in  design. 

BF-1,  BF-2,  and  BF-4  modifications  are  complete.  BF-3  will  not 
be  modified  (will  not  be  used  for  STOVL  Mode  4  operations). 
BF-4  has  resumed  Mode  4  operations.  Design  fix  is  on  BF-5 
and  subsequent  aircraft  and  new  configuration  is  to  full  life. 

BF-5 

LRIP  2 

2012 

Structure 

Roll  Control  Nozzle 
(RCN)  Doors 

Doors  separated  from  aircraft  BF-2  and 

BF-3  during  flight;  door  loads  not  well 
understood,  aero  pressures  higher  than 
expected.  Impact  not  limited  to  STOVL 
mode  operations  -  flight  not  to  exceed  400 
KCAS  below  1 8K ft  and  0.5  minimum  g-load. 

BF-2  and  BF-3  were  modified  with  an  interim  design, 
instrumented,  and  flown  to  verify  the  updated  loads  used 
to  develop  the  interim  and  final  design  doors.  The  Program 
Office  is  reviewing  a  redesign  to  support  production  in  LRIP  6. 

BF-38 

LRIP  6 

2014 

Structure 

3  Bearing  Swivel 
Nozzle  Door 

Door  attachment  wear/damage  found 
on  BF-1  (6/11)  requiring  new  inspection 
interval  every  25  Mode-4  (vertical-lift-fan- 
engaged)  flights.  During  Slow  Landing 
flight  testing,  measured  door  loads 
exceeded  limits. 

Interim  mod  complete  on  BF-1  and  BF-2,  instrumentation 
added  and  flight  test  is  ongoing.  Production  redesign  is  in 
progress. 

BF-44 

LRIP  7 

2015 

Structure 

Main  Landing  Gear 
(MLG)  Doors 

Door  cracking  observed  on  BF-1 ,  -2,  and  -4 
aft  door  adjacent  to  aft  lock. 

Instrumentation  added  to  BF-2  and  flight  loads  testing 
complete.  Models  correlated  and  root  cause  confirmed. 
Modification  of  the  rest  of  the  SDD  fleet  is  in  work;  production 
redesign  is  in  progress.  MLG  door  modification  will  be 
concurrent  with  AAID  modification. 

BF-44 

LRIP  7 

2015 

Propulsion 

Drive  Shaft 

Lift  fan  drive  shaft  undergoing  a  second 
redesign.  Original  design  inadequate 
due  to  shaft  stretch  requirements  to 
accommodate  thermal  growth,  tolerances, 
and  maneuver  deflections. 

Full  envelope  requirements  are  currently  being  met  on 
production  aircraft  with  an  interim  design  solution  using 
spacers  to  lengthen  the  early  production  drive  shaft.  Due  to 
the  heavy  maintenance  workload  associated  with  the  spacers, 
the  Program  Office  is  pursuing  an  improved  design  that  does 
not  require  class  spacers.  The  initial  improved  driveshaft 
design  failed  qualification  testing.  A  new  design  is  under 
development. 

BF-44 

LRIP  7 

2015 

Propulsion 

Clutch 

Lift  fan  clutch  has  experienced  higher  than 
expected  drag  heating  during  conventional 
(up  and  away)  flight. 

Testing  completed  to  determine  root  cause  of  drag  heating.  Fix 
includes  clutch  plate  width  reduction  on  LRIP  5  and  6  aircraft, 
at  the  expense  of  reduced  life  (engagements)  to  the  clutch. 

The  Program  Office  is  investigating  alternate  plate  material  to 
meet  engagement  requirement  on  subsequent  LRIPs. 

BF-44 

LRIP  7 

2015 

Propulsion 

Roll  Post  Nozzle 
Actuator 

Roll  post  nozzle  bay  temperatures  exceed 
current  actuator  capability.  Actuator  failure 
during  Mode  4  operations. 

Insulation  between  the  roll  post  nozzle  bay  and  the  actuator 
has  been  installed  and  testing  completed  through  the 

STOVL  flight  envelope.  All  LRIP  aircraft  have  been  fitted  with 
insulation  to  reduce  heat  transfer  into  the  bay  and  wear 
on  current  actuator.  A  newly  designed,  more  heat  tolerant 
actuator  is  scheduled  to  begin  testing  in  early  2013. 

TBD,  depending 
on  testing  and 
production  of 
redesigned 
actuator;  retrofit  of 
early  production 
fleet  will  occur  by 
attrition. 

Propulsion 

Bleed  Air  Leak 
Detectors 

Nuisance  overheat  warnings  to  the  pilot  are 
generated  because  of  poor  temperature 
sensor  design;  overheats  are  designed  to  be 
triggered  at  460  degrees  F,  but  have  been 
annunciated  as  low  as  340  degrees  F. 

More  accurate  temperature  sensors  in  the  bleed  air  leak 
detectors  have  been  designed  and  delivery  for  production 
aircraft  started  in  January  201 2. 

Detectors  on  early 
LRIP  aircraft  will 
be  replaced  by 
attrition. 

Propulsion 

Auxiliary  Air 

Inlet  Door  Aft 
down-lock  seal 
doors  (aka  "saloon 
doors") 

Doors  are  spring-loaded  to  the  closed 
position  and  designed  as  overlapping 
doors  with  a  0.5  inch  gap.  The  gap  induces 
air  flow  disturbance  and  make  the  doors 
prone  to  damage  and  out-of-sequence 
closing.  Damage  observed  on  BF-5. 

Seal  doors  are  being  redesigned  with  non-overlapping  doors 
and  stronger  spring  loads  to  ensure  proper  sequencing  and 
full  closure  of  the  doors. 

TBD 

F-35JSF  31 


DOD  PROGRAMS 


•  The  status  of  F-35B  door  and  propulsion  deficiencies  follows. 

-  The  upper  lift  fan  inlet  doors  continue  to  fail  to  operate 
correctly  due  to  poor  actuator  design.  Crews  have 
observed  failure  of  the  doors  to  close  on  flight  test  aircraft 
and  the  early  LRIP  aircraft  at  Eglin  AFB  during  ground 
operations.  Ground  maintenance  workaround  procedures 
are  in  place  to  ensure  correct  door  operation;  however, 
standard  maintenance  procedures  for  fleet  operations 

are  not  yet  in  place.  Newly  designed  actuators  will  not 
be  available  for  production  cut-in  until  BF-38,  a  Lot  6 
delivery  in  2014. 

-  Redesign  of  the  auxiliary  air  inlet  doors  is  complete.  The 
test  team  accomplished  flight  testing  of  the  aerodynamic 
loads  on  the  BF-1  doors  early  in  2012,  and  modified 

the  F-35B  static  test  article  with  the  new  auxiliary  air 
inlet  doors  in  August  2012  in  preparation  for  static  and 
durability  testing.  The  static  load  testing  was  completed  in 
mid-November,  followed  by  the  start  of  durability  testing. 
Results  of  the  testing  were  not  available  as  of  the  time  of 
this  report. 

-  Testing  and  analysis  continued  on  the  three-bearing  swivel 
nozzle  doors.  The  test  team  added  instrumentation  on 
BF-1  in  January  to  assess  the  dynamic  loads  on  the  door 
to  support  an  engineering  redesign.  BF-2  was  modified 
and  flight  testing  of  the  design  is  ongoing  as  of  the  time  of 
this  report.  Redesign  for  both  the  production  cut-in  and 
the  retrofit  plans  is  in  review  at  the  Program  Office.  Fleet 
restrictions  will  remain  in  effect  (slow  landings  below 
100  KCAS  are  prohibited)  until  the  program  modifies  the 
nozzle  doors. 

-  Temperatures  in  the  roll  control  nozzle  actuator  area 
exceeded  the  heat  tolerance  of  the  current  actuator  design 
during  flight  test,  necessitating  a  redesign.  The  program 
is  changing  the  insulation  in  the  nozzle  actuator  area  as  an 
interim  fix  and  redesigning  the  nozzle  actuator  to  improve 
heat  tolerance.  The  program  plans  to  begin  testing  the 
newly  designed  nozzle  actuator  in  early  2013. 

-  After  roll  control  nozzle  doors  separated  in-flight  in  2011, 
additional  testing  of  the  aerodynamic  loads  on  the  doors 
led  to  a  door  redesign.  A  production  redesign  currently 
under  review  with  the  Program  Office  increases  the  closing 
forces  on  the  door  to  prevent  aerodynamic  loads  opening 
and  possibly  damaging  doors  or  causing  door  separation. 

-  The  material  solution  to  unacceptably  high  clutch 
temperatures  observed  during  developmental  testing  is  to 
reduce  the  width  of  the  clutch  plates  in  later  LRIP  aircraft 
with  the  expectation  of  reducing  the  drag  and  associated 
heating  during  all  modes  of  flight.  Clutch  temperatures  are 
monitored  by  aircraft  sensors,  which  alert  the  pilot  when 
normal  temperature  limits  are  exceeded.  The  associated 
pilot  procedures  to  reduce  high  clutch  temperatures  require 
changing  flight  regimes  to  a  cooling  envelope  of  lower 
altitude  (below  11,000  feet)  and  lower  airspeed  (less  than 
280  knots);  such  a  procedure  during  combat  missions 
would  likely  increase  the  vulnerability  to  threats  and  cause 


the  pilot  to  abort  the  mission.  Further,  a  vertical  landing 
under  high  clutch  temperature  conditions  needs  to  be 
avoided  if  possible,  making  return  to  forward  basing  or 
ship-borne  operations  in  the  combat  zone,  where  a  vertical 
landing  would  be  required,  not  practical. 

-  The  program  added  spacers  to  the  lift  fan  driveshaft  to 
address  unanticipated  expansion/stretching  that  takes 
place  during  flight.  This  is  an  interim  solution  while  the 
program  redesigns  the  driveshaft  for  better  performance 
and  durability. 

•  Weight  management  of  the  F-35B  aircraft  is  critical  to 
meeting  the  Key  Performance  Parameters  (KPPs)  in  the 
ORD,  including  the  vertical  lift  bring-back  requirement.  This 
KPP  requires  the  F-35B  to  be  able  to  fly  an  operationally 
representative  profile  and  recover  to  the  ship  with  the 
necessary  fuel  and  balance  of  unexpended  weapons  (two 
1,000-pound  bombs  and  two  AIM-120  missiles)  to  safely 
conduct  a  vertical  landing. 

-  Weight  reports  for  the  F-35B  have  varied  little  in 
2012,  increasing  14  pounds  from  either  changes 
in  the  manufacturing  processes  or  more  fidelity  in 
the  weight  estimate.  Current  estimates  are  within 
231  pounds  (0.71  percent)  of  the  not-to-exceed  weight 
of  32,577  pounds  -  the  target  weight  of  the  aircraft  in 
January  2015  to  meet  specification  requirements  and 
ORD  mission  performance  requirements  for  vertical  lift 
bring-back.  The  small  difference  between  the  current 
weight  estimate  and  the  not-to-exceed  weight  allows  for 
weight  growth  of  0.32  percent  per  year. 

-  Managing  weight  growth  with  such  small  margins  will 
continue  to  be  a  significant  program  challenge.  Since 
the  program  will  conduct  the  technical  performance 
measurement  of  the  aircraft  in  January  2015,  well  before 
the  completion  of  SDD,  continued  weight  growth  through 
the  balance  of  SDD  will  affect  the  ability  of  the  F-35B 

to  meet  the  STOVL  mission  performance  KPP  during 
IOT&E.  Additionally,  production  aircraft  are  weighed  as 
part  of  the  government  acceptance  process,  and  the  early 
LRIP  lot  F-35B  aircraft  were  approximately  150  pounds 
heavier  than  the  predicted  values  found  in  the  weight  status 
report. 

•  The  program  announced  an  intention  to  change  performance 
specifications  for  the  F-35B,  reducing  turn  performance  from 
5.0  to  4.5  sustained  g’s  and  extending  the  time  for  acceleration 
from  0.8  Mach  to  1.2  Mach  by  16  seconds.  These  changes 
were  due  to  the  results  of  air  vehicle  performance  and  flying 
qualities  evaluations. 

•  Other  discoveries  included: 

-  As  with  the  F-35A,  horizontal  tail  surfaces  are 
experiencing  higher  than  expected  temperatures  during 
sustained  high-speed/high-altitude  flight,  resulting  in 
delamination  and  scorching  of  the  surface  coatings 
and  structure.  The  program  modified  the  tail  surfaces 
of  BF-2  in  September  to  permit  flight  testing  at  higher 
airspeeds.  The  coatings  delaminated  during  flight, 
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however,  suspending  further  flight  testing  in  the  higher 
airspeed  envelope  until  a  new  plan  for  the  coatings  can  be 
developed. 

-  Fuel  dump  testing  is  ongoing  on  BF-4  after  a  redesign  of 
seals  on  the  lower  trailing  edge  flaps.  Previous  testing 
with  the  original  seals  resulted  in  fuel  penetrating  the  cove 
area  behind  the  flaps  and  wetting  the  fuselage,  allowing 
fuel  to  pool  near  the  Integrated  Power  Package  exhaust 
where  the  fuel  is  a  fire  hazard.  Testing  with  the  new  seals 
has  shown  less  fuel  penetration  with  flaps  fully  retracted 
and  with  flaps  extended  to  20  degrees;  however,  fuel  traces 
inside  the  flaperon  cove  were  observed  during  post-flight 
inspections.  The  test  team  is  also  testing  redesigned  exit 
nozzles  of  different  shape  and  cross-sectional  areas.  As 
of  the  end  of  November  2012,  11  relevant  test  flights  have 
been  accomplished;  more  flights  will  be  necessary  to 
resolve  the  deficiency. 

-  Planned  wet  runway  testing,  required  to  assess  braking 
performance  with  a  new  brake  control  unit,  has  been 
delayed  due  to  the  inability  to  create  the  properly  degraded 
friction  conditions  at  the  Patuxent  River  Naval  Air  Station 
(NAS),  Maryland.  The  F-35B  training  aircraft  at  Eglin 
will  be  restricted  to  dry  runway  operations  only  until  the 
wet  runway  testing  is  completed. 

F-35C  Flight  Sciences 

Flight  Test  Activity  with  CF-1,  CF-2,  and  CF-3  Test  Aircraft 

•  F-35C  flight  sciences  focused  on: 

-  Verification  of  the  basic  flight  envelope  for  the  first 
production  F-35C  aircraft 

-  Expansion  of  the  flight  envelope  with  external  weapons 
loaded  on  the  aircraft  (AIM-9X  short-range  missile  in 
subsonic  flight) 

-  Testing  of  arresting  hook  system  modifications 

-  Preparing  for  executing  carrier  landings  in  the  simulated 
carrier  environment  at  Lakehurst  Naval  Test  Facility,  New 
Jersey,  including  handling  qualities  at  approach  speeds  at 
carrier  landing  weights 

-  Surveying  handling  qualities  in  the  transonic  flight  regimes 

-  Regression  testing  of  new  air  vehicle  systems  software 

•  As  of  November,  the  test  team  executed  a  sortie  rate  of 

7. 1  sorties  per  aircraft  per  month  compared  to  the  goal  of  6.4. 
The  program  accomplished  1 10  percent  of  the  planned  F-35C 
flight  sciences  sorties,  completing  233  vice  211  planned 
through  the  end  of  November. 

•  The  program  plans  to  deliver  the  final  F-35C  flight  sciences 
aircraft,  CF-5,  in  late  2012,  followed  soon  by  the  first 
production  F-35C  from  Lot  4.  CF-5  flew  its  first  company 
acceptance  flight  at  the  end  of  November. 

Flight  Sciences  Assessment 

•  The  program  completed  80  percent  of  the  baseline  test  points 
planned  though  November  2012,  accomplishing  1,060  test 
points  of  a  planned  1,327.  Flight  restrictions  blocked 
accomplishment  of  a  portion  of  the  planned  baseline  test 
points  until  a  new  version  of  vehicle  systems  software  became 
available. 


•  The  test  team  flew  an  additional  253  test  points  from  flight 
test  requests  and  pulled  896  test  points  forward  from  work 
planned  for  2013. 

•  By  accomplishing  envelope  test  points  planned  for  completion 
in  later  years,  the  test  team  was  able  to  keep  ahead  of  the 
cumulative  SDD  test  point  objectives,  as  was  the  case  in 
F-35A  and  F-35B  flight  sciences.  While  awaiting  new  vehicle 
systems  software  required  to  complete  planned  envelope 
testing  in  2012,  the  test  team  accomplished  points  in  other 
areas  of  the  flight  envelope.  For  F-35C  flight  sciences,  the  test 
team  had  accomplished  116  percent  of  the  planned  number  of 
cumulative  test  points  scheduled  for  completion  by  the  end  of 
November  (4,330  cumulative  points  accomplished  against  a 
goal  of  3,748  points). 

•  Weight  management  of  the  F-35C  variant  is  important  for 
meeting  air  vehicle  performance  requirements.  F-35C  weights 
have  generally  decreased  in  the  monthly  estimates  during 
2012.  The  latest  weight  status  report  from  November  2012 
showed  the  estimated  weight  of  34,522  pounds  to  be  within 
346  pounds  (1.0  percent)  of  the  projected  maximum  weight 
needed  to  meet  technical  performance  requirements  in 
January  2016.  This  margin  allows  for  0.31  percent  weight 
growth  per  year.  The  program  will  need  to  continue  rigorous 
weight  management  through  the  end  of  SDD  to  avoid 
performance  degradation  and  operational  impacts. 

•  The  program  announced  an  intention  to  change  performance 
specifications  for  the  F-35C,  reducing  turn  performance 
from  5.1  to  5.0  sustained  g’s  and  increasing  the  time 

for  acceleration  from  0.8  Mach  to  1.2  Mach  by  at  least 
43  seconds.  These  changes  were  due  to  the  results  of  air 
vehicle  performance  and  flying  qualities  evaluations. 

•  Discoveries  included: 

-  Due  to  the  difference  in  wing  design,  transonic  buffet 
becomes  severe  in  different  portions  of  the  flight  envelope 
and  is  more  severe  in  the  F-35C  than  the  other  variants. 

The  program  is  making  plans  for  investigating  how  to 
reduce  the  impact  of  transonic  roll  off  in  the  F-35C  with 
the  use  of  wing  spoilers;  however,  detailed  test  plans  are 
not  complete. 

-  As  with  the  F-35A  and  F-35B,  horizontal  tail  surfaces  are 
experiencing  higher  than  expected  temperatures  during 
sustained  high-speed/high-altitude  flight,  resulting  in 
delamination  and  scorching  of  the  surface  coatings  and 
structure.  In  August,  the  test  team  installed  new  coatings 
on  CF-1  horizontal  tails,  designed  to  prevent  scorching 
and  delaminating  during  prolonged  use  of  afterburner 
pursuing  high  airspeed  test  points.  However,  portions  of 
the  coatings  dis-bonded  during  flight,  suspending  further 
testing  of  the  high  airspeed  portion  of  the  envelope. 

-  The  test  team  investigated  alternative  trailing  edge  flap 
settings  to  improve  flying  qualities  during  carrier  landing 
approach.  While  pilot  surveys  showed  handling  qualities 
were  improved  with  a  15-degree  flap  setting,  flight  test 
data  to  date  have  shown  that  30  degrees  of  flaps  are 
required  to  meet  the  KPP  for  maximum  approach  speed  of 
145  knots  at  required  carrier  landing  weight. 
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Mission  Systems 

Flight  Test  Activity  with  AF-3,  AF-6,  AF-7,  BF-17,  and  BF-18 

Test  Aircraft  and  Software  Development  Progress 

•  Mission  systems  are  developed  and  fielded  in  incremental 
blocks  of  capability. 

-  Block  1.  The  program  designated  Block  1  for  initial 
training  capability  and  allocated  two  increments: 

Block  1 A  for  Lot  2  (12  aircraft)  and  Block  IB  for  Lot  3 
aircraft  (17  aircraft).  No  combat  capability  is  available  in 
either  Block  1  increment.  (Note:  Remaining  development 
and  testing  of  Block  0.5  initial  infrastructure  was  absorbed 
into  Block  1  during  the  program  restructuring  in  2011.) 

-  Block  2A.  The  program  designated  Block  2A  for 
advanced  training  capability  and  associated  this  block 
with  Lots  4  and  5.  No  combat  capability  is  available  in 
Block  2A. 

-  Block  2B.  The  program  designated  Block  2B  for  initial, 
limited  combat  capability  for  selected  internal  weapons 
(AIM-120C,  GBU-32/31,  and  GBU-12).  This  block 

is  not  associated  with  the  delivery  of  any  production 
aircraft.  Block  2B  software  will  be  used  to  retrofit  earlier 
production  aircraft. 

-  Block  3i.  Block  3i  is  Block  2A  capability  re-hosted  on  an 
improved  integrated  core  processor  for  Lots  6  through  8. 

-  Block  3F.  The  program  designated  Block  3F  as  the  full 
SDD  capability  for  production  Lot  9  and  later. 

•  The  Patuxent  River  test  site  accepted  two  early  production 
aircraft  from  Lot  3  (BF-17  and  BF-18)  to  support  mission 
systems  development  and  testing,  in  accordance  with 
guidance  following  the  Technical  Baseline  Review  (TBR)  in 
October  2010.  Aircraft  BF-17  ferried  to  Patuxent  River  on 
October  4th  and  BF-18,  on  November  8th.  BF-17  began  radar 
signature  testing  soon  after  arrival;  BF-18  has  yet  to  fly  test 
sorties. 

•  The  four  mission  systems  flight  test  aircraft,  three  assigned 
to  the  Edwards  AFB  test  center,  and  one  BF-17  assigned 
to  Patuxent  River,  flew  an  average  rate  of  5.0  sorties  per 
aircraft  per  month  through  November,  exceeding  the  planned 
rate  of  4.4  by  14  percent.  Mission  systems  test  aircraft  flew 
115  percent  of  the  test  flights  planned  through  the  end  of 
November  (222  sorties  completed  compared  to  193  planned). 

•  The  test  team  accomplished  95  percent  of  the  planned  2012 
baseline  test  points  by  the  end  of  November  (1,238  baseline 
test  points  accomplished,  1,308  planned).  The  team  also 
accomplished  an  additional  610  test  points  for  regression 
testing  of  additional  revisions  of  Block  2A  software. 

Mission  Systems  Assessment 

•  The  program  made  limited  progress  in  2012  in  fielding 
capability,  despite  relatively  high  sortie  and  test  point 
completion  rates. 

-  Software  delivery  to  flight  test  was  behind  schedule  or  not 
complete  when  delivered. 

■  Block  1  software  has  not  been  completed;  approximately 
20  percent  of  the  planned  capability  has  yet  to  be 
integrated  and  delivered  to  flight  test. 


■  The  first  version  of  Block  2A  software  was  delivered 
four  months  late  to  flight  test.  In  eight  subsequent 
versions  released  to  flight  test,  only  a  limited  portion 
of  the  full,  planned  Block  2A  capability  (less  than 
50  percent)  became  available  and  delivered  to 
production.  Block  2A  has  no  combat  capability. 

■  Block  2B  software  was  planned  to  be  delivered  to  flight 
test  by  the  end  of  2012,  but  less  than  10  percent  of  the 
content  was  available  for  integration  and  testing  as  of 
the  end  of  August.  A  very  limited  Block  IB  software 
version  was  delivered  to  the  Cooperative  Avionics  Test 
Bed  aircraft  in  early  November  for  integration  testing. 

■  The  program  made  virtually  no  progress  in  the 
development,  integration,  and  laboratory  testing  of  any 
software  beyond  2B.  Block  3i  software,  required  for 
delivery  of  Lot  6  aircraft  and  hosted  on  an  upgraded 
processor,  has  lagged  in  integration  and  laboratory 
testing. 

-  The  test  team  completed  1,238  (95  percent)  of  the  planned 
1,308  baseline  test  points  by  the  end  of  November. 

The  team  also  completed  an  additional  610  points  for 
regression  of  multiple  versions  of  software.  Although  the 
test  team  accomplished  test  points  in  2012  as  planned, 
little  flight  testing  of  advanced  mission  systems  capability 
has  taken  place.  Additionally,  current  planning  of 
baseline  test  points  results  in  shortfalls  in  production 
aircraft  capabilities  that  will  persist  into  2014.  Only 
2,532  (23  percent)  of  the  10,966  total  mission  systems  test 
points  planned  for  SDD  have  been  accomplished  as  of  the 
end  of  November  2012.  Of  those  completed,  54  percent 
supported  testing  of  basic  mission  systems  capabilities, 
such  as  communications,  navigation,  and  basic  radar 
functions,  with  the  remaining  46  percent  being  comprised 
of  radar  signature  testing  (which  does  not  involve  or 
require  any  mission  systems  capability),  software  maturity 
demonstrations,  and  verification  of  capabilities  for  early 
production  aircraft  delivery. 

•  Although  all  Lot  2  and  Lot  3  aircraft  -  in  the  Block  1 
configuration  -  were  either  delivered  to  the  Services  or 
awaiting  final  delivery  as  of  the  time  of  this  report,  the  test 
team  had  accomplished  only  54  percent  (738)  of  the  1,371  test 
points  in  the  original  Block  1  test  plan.  This  resulted  in  the 
Lot  2  and  Lot  3  aircraft  being  accepted  by  the  Services  with 
major  variances  against  the  expected  capabilities  and  added  to 
a  bow  wave  of  test  points  that  will  have  to  be  completed  in  the 
future. 

-  For  example,  when  six  F-35A  and  six  F-35B  Lot  2  aircraft 
were  delivered  to  the  training  center  in  the  Block  1A 
configuration,  only  37  of  51  Block  1 A  capabilities  on 
contract  were  delivered.  Subsequently,  the  program 
delivered  ten  Lot  3  aircraft  to  the  training  center  in 

2012  in  a  partial  Block  IB  configuration  (three  F-35As, 
five  F-35Bs,  and  two  F-35Bs  produced  for  the  United 
Kingdom). 
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-  The  Block  IB  configuration  was  designed  to  provide  an 
additional  35  capabilities;  however,  the  program  delivered 
only  10  prior  to  the  delivery  of  the  first  Lot  3  aircraft.  The 
program  is  in  the  process  of  upgrading  Block  1 A  aircraft  to 
the  IB  configuration;  however,  no  additional  capabilities 
were  delivered  with  the  Block  IB  configuration.  Examples 
of  expected  capabilities  that  were  not  delivered  include  air 
vehicle  and  off-board  prognostic  health  management  tools, 
instrument  landing  system  (ILS)  for  navigation,  distributed 
aperture  system  (DAS)  video  displaying  in  the  helmet, 
corrosion  data  recording  capability,  and  night  vision 
imaging  integration  with  the  helmet. 

•  The  Services  began  accepting  Lot  4  aircraft  in  the  Block  2A 
configuration  in  November  with  major  variances  against 
the  expected  capabilities.  The  program  plans  to  continue 

to  develop  and  test  the  incomplete  and  remaining  Block  2A 
capabilities  using  incremental  versions  of  Block  2  A  software 
and  update  the  aircraft  previously  delivered  with  partial 
capabilities  in  late  2013.  The  continued  development 
and  testing  of  Block  2A  software  will  be  accomplished 
concurrently  with  the  Block  2B  software  capabilities. 

•  Simultaneous  development  of  new  capabilities,  associated 
with  the  next  blocks  of  software,  competes  with  the  flight  test 
resources  needed  to  deliver  the  scheduled  capability  for  the 
next  lot  of  production  aircraft. 

-  For  example,  the  testing  needed  for  completion  of  the 
remaining  20  percent  of  Block  1  capabilities  and  50 
percent  of  Block  2  A  capabilities  will  have  to  be  conducted 
while  the  program  is  introducing  Block  2B  software  to 
flight  test.  Software  integration  tasks  supporting  Block  2B 
(and  later  increments)  were  delayed  in  2012  as  contractor 
software  integration  staff  were  needed  to  support  Block  2A 
development,  test,  and  anomaly  resolution. 

-  This  process  forces  the  program  to  manage  limited 
resources,  including  the  software  integration  labs,  the 
cooperative  avionics  test  bed  aircraft,  and  the  mission 
systems  test  aircraft,  to  address  the  needs  of  multiple 
versions  of  software  simultaneously.  The  demand  on  flight 
test  to  complete  test  points  for  verification  of  capability 
for  production  software  releases,  while  simultaneously 
accomplishing  test  points  for  expanding  development  of 
capability  will  continue  to  challenge  the  test  team  and  add 
to  the  inherent  concurrency  of  the  program.  The  program 
intends  Block  3i  to  enter  flight  test  in  mid-2013,  which 
will  be  conducted  concurrently  with  the  final  15  months 
of  Block  2B  flight  test.  The  program  intends  for  Block  3F 
to  enter  a  3  3 -month  developmental  flight  test  period  in 
early  2014. 

•  Recognizing  the  burden  and  challenges  caused  by  the 
concurrency  of  production  and  flight  test,  the  Program  Office 
is  developing  a  capability  management  plan  and  review  board 
to  evaluate  priorities  and  trades  of  capabilities  within  blocks 
and  for  deferral  out  of  SDD  if  necessary. 

•  Shortfalls  in  the  test  resources  required  to  test  mission 
systems  electronic  warfare  capabilities  under  operationally 


realistic  conditions  were  identified  by  DOT&E  in  February. 
The  needed  resources  and  funding  were  being  considered  by 
the  Department  at  the  time  of  this  report. 

•  Discoveries  included: 

-  The  test  team  continued  to  work  through  technical 
problems  with  the  helmet-mounted  display  system, 
which  is  deficient.  The  program  was  addressing  five 
problems  at  the  time  of  this  report.  Jitter,  caused  by 
aircraft  vibrations  and  exacerbated  by  aircraft  buffet, 
makes  the  displayed  information  projected  to  the  pilot 
hard  to  read  and  unusable  under  certain  flight  conditions. 
Night  vision  acuity  is  not  meeting  specification 
requirements.  Latency  of  the  projected  imagery  from  the 
DAS  is  currently  down  to  133  milliseconds,  below  the 
human  factors  derived  maximum  of  150  milliseconds, 
but  still  requires  additional  testing  to  verify  adequacy. 
Boresight  alignment  between  the  helmet  and  the 
aircraft  is  not  consistent  between  aircraft  and  requires 
calibration  for  each  pilot.  Finally,  a  recently  discovered 
technical  problem  referred  to  as  “green  glow”  has 

been  experienced  when  light  from  the  cockpit  avionics 
displays  leaks  into  the  helmet-mounted  display  and 
degrades  visual  acuity  through  the  helmet  visor  under 
low  ambient  light  conditions.  The  test  team  is  planning 
additional,  dedicated  ground  and  flight  testing  to  address 
these  technical  problems. 

-  Electronic  warfare  antenna  performance  of  the  first  three 
production  lots  of  aircraft  was  not  meeting  contract 
specification  requirements.  Poorly  designed  connectors 
created  signal  distortion  in  the  six  antenna  apertures 
embedded  in  the  aircraft.  The  Program  Office  determined 
that  3 1  aircraft  are  affected  and  require  additional  testing 
of  each  antenna.  Testing  of  the  apertures  began  on 

SDD  aircraft  at  Edwards  AFB  in  November.  Progress 
in  verifying  the  performance  of  the  electronic  warfare 
system  will  be  affected  until  additional  testing  of  the 
apertures  in  the  aircraft  is  completed  and  any  necessary 
retrofits  accomplished  on  the  mission  systems  test 
aircraft. 

-  Helmet-mounted  display  video  imagery  needed  to 
successfully  analyze  and  complete  portions  of  the 
mission  systems  test  plans  cannot  be  reliably  recorded  on 
either  the  portable  memory  device  or  the  data  acquisition 
recording  and  telemetry  pod.  The  program  began  testing 
fixes  in  August.  Until  resolved,  the  overall  impact 

is  336  total  mission  systems  test  points  that  are  not 
achievable. 

-  The  program  projects  utilization  rates  for  the  two 
processors  that  support  the  panoramic  cockpit  display 
to  be  greater  than  100  percent  when  assessed  against 
Block  3  capabilities.  The  program  initiated  plans  to 
optimize  the  core  processor  software  to  reduce  these 
rates. 

-  The  program  is  tracking  mission  system  software 
stability  by  analyzing  the  number  of  anomalies 
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observed  as  a  function  of  flight  time.  Current  program 
objectives  for  early  mission  system  software  in 
flight  test  are  to  have  integrated  core  processor  and 
Communications/Navigation/Identification  Friend  or  Foe 
(CNI)  anomaly  rates  be  15  hours  or  more  between  events. 
Recent  reports  for  the  latest  mission  systems  software  in 
flight  test  -  version  2AS2.8  -  show  a  rate  of  6.3  hours 
between  anomalies  based  on  88  hours  of  flight  test. 

Weapons  Integration 

•  Weapons  integration  includes  flight  sciences,  mission  systems, 
and  ground  maintenance  support.  Testing  includes  measuring 
the  environment  around  the  weapon  during  carriage  (internal 
and  external),  handling  characteristics  of  the  aircraft,  safe 
separation  of  the  weapon  from  the  aircraft,  and  weapons 
delivery  accuracy  events. 

•  In  2012,  the  program  conducted  detailed  planning  of  the 
weapons  integration  events  necessary  to  complete  SDD. 

This  planning  yielded  a  schedule  for  completing  weapons 
integration  for  Block  2B  and  Block  3F  combat  capability. 

•  The  test  team  conducted  the  flight  sciences  loads,  flutter,  and 
environmental  testing  necessary  to  certify  a  limited  Block  2B 
carriage  envelope  of  the  F-35A  and  F-35B  aircraft  for  Joint 
Direct  Attack  Munition,  GBU-12  laser  guided  bomb,  and 
the  AIM-120  air-to-air  missile  to  enable  the  start  of  active 
flight  testing.  As  of  the  end  of  October,  this  testing  had 
achieved  captive  carriage  and  first  safe  separation  of  an  inert 
AIM- 120  missile  (on  the  A  model)  and  inert  Joint  Direct 
Attack  Munition  (on  both  the  A  and  B  model).  However,  to 
date,  weapons  integration  has  been  limited  by  the  following 
deficiencies: 

-  Instrumentation 

-  Data  recording  shortfalls 

-  Deficient  mission  systems  performance  in  radar, 
Electro-Optical  Targeting  System  (EOTS),  fusion,  and  the 
helmet 

-  Lack  of  radar  fusion  support  to  the  AIM- 120  air-to-air 
missile 

-  EOTS  inability  to  accurately  track  and  designate  targets  for 
the  air-to-ground  munitions, 

-  Deficient  fused  situational  awareness  presentation  to  the 
pilot 

•  The  successful  execution  of  the  detailed  schedule  developed 
this  year  was  dependent  on: 

-  The  ability  of  the  program  to  deliver  mission  systems 
capability  required  to  start  weapons  integration  in 
April  2012 

-  Adequate  margin  in  the  test  schedule  to  accommodate 
repeated  testing,  cancellations  due  to  weather,  range  assets, 
and  operational  support 

-  Reliable  instrumentation  and  range  support 

•  None  of  these  assumptions  have  proven  true,  adding  risk  to 
the  execution  of  the  overall  schedule.  Deferrals  of  mission 
systems  capabilities  to  later  blocks  and  delays  for  corrections 
to  test  instrumentation  and  data  recording  have  removed  the 


schedule  margins.  The  impact  of  these  delays  will  potentially 
require  an  additional  18  months  added  to  the  schedule  for 
weapons  integration  events. 

Static  Structural  and  Durability  Testing 

•  Durability  testing  on  the  ground  test  articles  of  all  three 
variants  continued  in  2012;  progress  is  measured  in  aircraft 
lifetimes.  An  aircraft  lifetime  is  defined  as  8,000  Equivalent 
Flight  Hours  (EFH),  which  is  a  composite  of  time  under 
different  test  conditions  (i.e.,  maneuver  and  buffet  for 
durability  testing).  In  accordance  with  the  SDD  contract,  all 
three  variants  will  complete  two  full  lifetimes,  or  16,000  EFH 
of  durability  testing.  The  completion  dates  for  the  second 
aircraft  lifetimes  are  late  2014  for  the  F-35B  and  early  2015 
for  the  F-35A  and  F-35C.  Plans  for  a  third  lifetime  of 
durability  testing  for  all  three  variants  are  under  development. 

•  The  F-35A  ground  test  article,  AJ-1,  completed  the  first  of 
two  planned  aircraft  lifetimes  in  August,  as  planned.  F-35A 
durability  testing  continued  into  the  second  planned  aircraft 
lifetime  at  the  time  of  this  report,  completing  9,117  EFH  as  of 
December  5,  2012. 

•  F-35B  durability  testing  on  BH-1  was  restarted  in  January  after 
a  16-month  break  caused  by  the  discovery,  analysis,  and  repair 
of  a  crack  in  a  wing  carry-through  bulkhead  at  1,055  EFH. 
Since  restarting,  an  additional  5,945  hours  of  testing  had  been 
completed  by  the  end  of  October,  bringing  the  total  test  time 

to  7,000  EFH  and  putting  the  testing  ahead  of  the  restructured 
2012  plan  to  complete  6,500  hours  by  the  end  of  the  year. 

•  F-35C  durability  testing  began  in  March  and  the  test  article, 
CJ-1,  had  completed  4,000  EFH  of  fatigue  testing  as  of 
October,  as  scheduled. 

•  Component  durability  testing  for  two  lifetimes  of  the  vertical 
tails  was  completed  for  the  F-35A  and  F-35B  during  2012. 

This  testing  was  started  in  August  for  the  F-35C.  Component 
testing  of  the  horizontal  tail  for  the  F-35C  completed 

8,000  EFH,  or  one  lifetime,  in  May,  and  an  additional 
2,000  EFH  by  the  end  of  October.  (Component  testing  of 
the  horizontal  tails  for  the  F-35A  and  F-35B  completed  two 
lifetimes  of  testing  in  2011.) 

•  The  program  redesigned  the  F-35B  auxiliary  air  inlet  doors, 
required  for  STOVL  operations,  and  began  flight  testing  in 
2012.  Redesigned  doors  have  been  installed  on  the  static  loads 
test  article  (BG-1)  and  completed  static  loads  testing  in  early 
November,  followed  by  the  start  of  durability  testing.  The 
report  from  the  static  testing  is  scheduled  to  be  completed  by 
the  end  of  2012;  however,  the  results  of  the  durability  testing 
are  not  scheduled  to  be  available  until  mid-2013.  The  program 
has  already  ordered,  received,  and  begun  installing  retrofit  kits 
for  the  auxiliary  air  inlet  door  modifications  on  fielded  Lot  4 
aircraft. 

•  Discoveries  from  durability  testing  included  significant 
findings  in  both  the  F-35A  and  F-35B  ground  test  articles. 

-  In  the  F-35A,  a  crack  was  discovered  on  the  right  wing 

forward  root  rib  at  the  lower  flange  (this  is  in  addition  to 
the  crack  found  and  reported  in  the  FY11  Annual  Report). 
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Also,  a  crack  was  found  by  inspection  in  the  right  hand 
engine  thrust  mount  shear  web  in  February.  Testing  was 
halted  while  the  crack  was  inspected  and  analyzed,  then 
restarted  to  complete  subsequent  blocks  of  testing. 

-  In  the  F-35B,  the  program  halted  testing  in  December  2012 
after  multiple  cracks  were  found  in  a  bulkhead  flange 
on  the  underside  of  the  fuselage  during  the  7,000-hour 
inspection.  Root  cause  analysis,  correlation  to  previous 
model  predictions,  and  corrective  action  planning  were 
ongoing  at  the  time  of  this  report.  Other  cracks  were 
previously  discovered  in  the  B-model  test  article;  one  on 
the  right  side  of  the  fuselage  support  frame  in  February  and 
one  at  a  wing  pylon  station  in  August,  both  of  which  were 
predicted  by  modeling.  Another  crack  in  the  shear  web  tab 
that  attaches  to  the  support  frame  was  discovered  in  March. 
Also,  excessive  wear  was  found  on  the  nose  landing  gear 
retractor  actuator  lugs  and  weapons  bay  door  hinges.  All 
of  these  discoveries  will  require  mitigation  plans  and  may 
include  redesigning  parts  and  additional  weight. 

•  The  results  of  findings  from  structural  testing  highlight  the 
risks  and  costs  of  concurrent  production  with  development. 

The  Program  Office  estimates  of  the  weight  changes  to 
accommodate  known  limited  life  parts  discovered  so  far  from 
structural  testing  are  shown  in  the  table  below.  These  weight 
increases  are  in  the  current  weight  status  reports  for  each  of 
the  variants.  Discoveries  during  the  remaining  two  years  of 
structural  testing  will  potentially  result  in  more  life-limited 
parts  and  associated  impacts  to  weight  and  design. 


Variant 

Number  of 
Life  Limited 
Parts 

Retrofit  Weight 
Increase  to  Early 
LRIP  Aircraft 
(prior  to  production 
cut-in) 

Production  Weight 
Increase 
(cut-in  varies  from 
LRIP  4  to  LRIP  7) 

F-35A 

19 

38  pounds 

20  pounds 

F-35B 

20 

123  pounds 

33  pounds 

F-35C 

7 

5  pounds 

1  pound 

Modeling  and  Simulation 

Verification  Simulation  (VSim) 

•  The  Verification  Simulation  (VSim)  is  a  man-in-the-loop, 
mission  software-in-the-loop  simulation  developed  to  meet 
the  operational  test  agencies’  requirements  for  Block  2B 
OT&E  and  Block  3  IOT&E. 

•  The  program  continued  detailed  technical  reviews  of  the 
VSim  with  the  contractor  and  subcontractors  supplying  its 
component  models  during  2012.  Sensor  model  reviews  took 
place  for  the  electronic  warfare,  radar,  and  DAS  infrared 
sensors.  The  program  held  similar  detailed  reviews  for  the 
inertial  navigation  system  (INS)  and  Global  Positioning 
System  (GPS)  models,  as  well  as  for  the  VSim  Battle  Space 
Environment  (BSE),  a  collection  of  background  environment 
models  with  which  the  sensor  and  navigation  system  models 
interact. 

•  At  the  time  of  this  report,  the  program  was  tracking  1 1  formal 
risks  with  regard  to  VSim,  4  of  them  characterized  as  high 


risk,  the  other  7  characterized  as  moderate.  These  1 1  risks  fall 
into  4  general  categories: 

-  Risks  associated  with  timeliness  of  VSim  software 
delivery,  completeness  with  regard  to  modeled  capabilities, 
and  discrepancies  between  VSim  and  aircraft  software  due 
to  mismatches  in  the  software  versions  that  are  current  in 
VSim  and  those  that  are  current  in  the  aircraft  at  any  given 
time. 

-  Risks  associated  with  the  timeliness,  completeness,  and 
production-representativeness  of  data  from  flight  testing 
and  other  testing  used  to  verify  and  validate  VSim. 

-  Risks  regarding  the  time  and  manpower  needed  to 
analyze  VSim  validation  data  and  perform  accreditation 
assessments. 

-  Fundamental  risks  regarding  the  ability  of  VSim  to 
faithfully  replicate  all  aspects  of  F-35  and  threat  systems 
performance. 

•  In  addition  to  the  risks  cited  by  the  Program  Office,  DOT&E 
has  highlighted  shortfalls  in  the  test  resources  needed  to 
gather  key  elements  of  data  required  for  validation  of  the 
VSim  for  IOT&E,  in  particular  for  electronic  warfare 
performance.  These  shortfalls  are  a  function  of  limitations  in 
the  test  assets  currently  available  to  represent  threat  systems. 
DOT&E  has  made  formal  recommendations  to  address  the 
shortfalls. 

Other  Models  and  Corporate  Labs  Activity 

•  The  Program  Office  has  accredited  7  of  the  28  models  and 
simulations  currently  planned  to  support  verification  of  the 
F-35. 

•  The  program  accredited  three  models  intended  for  use  in 
contract  specification  verification  in  2012.  These  are  the 
Ejection  Seat  Model,  the  Support  Enterprise  Model  (SEM), 
and  the  Automatic  Dynamic  Analysis  of  Mechanical  Systems 
(ADAMS)  model.  A  fourth  model.  Prognostic  Health 
Management  (PHM)  Coverage  Analysis  Tool  (PCAT),  is  in 
final  accreditation  review  at  the  Program  Office  at  the  time  of 
this  report. 

-  The  Ejection  Seat  Model  is  used  to  verify  the  terrain 
clearance  requirements  of  the  F-35  ejection  seat  under 
different  flight  conditions. 

-  SEM  is  used  to  assess  the  logistics  infrastructure 
requirements  of  the  fielded  F-35  Air  System. 

-  ADAMS  is  used  to  assess  weapon  store-to-aircraft 
clearances  and  interfaces  during  loading,  carriage  and 
separation,  evaluating  weapon  arming  and  de-arming,  and 
other  weapons  system  separation  functions. 

-  PCAT  is  a  spreadsheet-based  application  that  rolls-up 
probabilities  of  fault  isolation  and  fault  detection  to 
various  line  replaceable  units. 

•  The  program  plans  to  accredit  6  models  and  simulations 
intended  for  use  in  requirements  verification  plan  in  2013, 
with  the  remaining  15  accreditations  due  between  2014  and 
the  end  of  SDD  in  2017. 

•  The  Program  Office  has  identified  challenges  for  2013  with 
respect  to  obtaining  and  analyzing,  in  a  timely  fashion,  the 
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validation  data  needed  to  accredit  the  GPS  System  Model 
Simulation  (GSMS)  and  Modeling  System  for  Advanced 
Investigation  of  Countermeasures  (MOSAIC)  infrared 
countermeasures  effectiveness  models. 

•  In  2011,  the  Air  Force  airworthiness  authorities  identified  the 
pilot  escape  system  installed  in  the  early  LR1P  aircraft  as  a 
serious  risk.  Validation  of  expected  performance  of  the  F-35 
escape  system  is  supported  by  modeling  the  ejection  seat  as 
well  as  the  effectiveness  of  the  transparency  removal  system 
for  the  canopy  during  the  ejection  sequence. 

-  For  the  ejection  seat  model,  the  program  used  data 
from  sled  testing  under  straight  and  level  conditions  to 
predict  performance  of  the  ejection  seat  under  non-zero 
angle-of-bank  (including  inverted)  conditions.  Interactions 
between  the  pilot,  the  ejection  seat,  and  the  canopy  during 
the  ejection  sequence,  however,  are  not  well  understood, 
particularly  during  other  than  straight  and  level  ejection 
conditions. 

-  Testing  of  the  transparency  removal  system  under 
off-nominal  conditions  to  better  understand  these 
interactions  was  scheduled  for  March  2012.  The  program 
expects  this  testing  to  take  place  in  December  2012. 

Training  System 

•  The  program  initiated  flight  operations  at  the  Integrated 
Training  Center,  Eglin  AFB,  Florida,  in  2012  with  both  the 
F-35  A  and  F-35B  aircraft. 

-  The  Air  Force  accepted  six  F-35  A  aircraft  from  production 
Lot  2  in  2011  at  Eglin  in  the  Block  1 A  configuration,  but 
did  not  commence  flight  operations  until  March  2012 
when  the  Air  Force  airworthiness  authorities  provided 

the  necessary  flight  clearance,  which  limited  operations 
to  previously  qualified  F-35  pilots.  In  July,  the  Air  Force 
changed  the  flight  clearance  to  allow  pilots  not  previously 
qualified  to  fly  at  Eglin,  which  paved  the  way  for  F-35  A 
pilot  training  to  begin  later  in  the  year. 

-  The  program  delivered  six  F-35B  aircraft  from  production 
Lot  2  to  Eglin  between  January  and  May  2012.  Also  in 
May,  Navy  airworthiness  authorities  provided  a  flight 
clearance  for  F-35B  flight  operations  to  begin  at  Eglin. 

-  The  program  added  10  production  Lot  3  aircraft  -  all 
in  the  Block  IB  configuration  -  to  Eglin  by  the  end  of 
October  2012  to  support  flight  training:  3  F-35A  aircraft 
between  July  and  August  and  7  F-35B  aircraft  (5  for  the 
Marine  Corps  and  2  for  the  United  Kingdom)  between  July 
and  October.  These  deliveries  were  later  than  planned  due 
to  late  availability  of  an  adequate  Autonomous  Logistics 
Information  System  (AL1S)  at  Eglin  to  support  the 
Block  IB  aircraft  configuration. 

•  In  July  2012,  DOT&E  recommended  to  the  Air  Force,  the 
operational  test  agencies,  and  the  JSF  Program  Office  that 
the  training  OUE  be  delayed  until  the  system  matures  and 
possesses  some  combat  capability  relevant  to  an  operational 
evaluation. 

-  DOT&E  identified  seven  indicators  which  highlighted  a 
lack  of  overall  system  maturity:  abort  rates  higher,  and 


trending  flat,  than  the  Air  Force  risk  assessment  identified 
for  a  maturing  system;  the  trend  in  discovery  as  indicated 
by  the  rate  of  new  Deficiency  Reports;  the  high  number 
of  “workarounds”  needed  to  support  maintenance  and 
sortie  generation  activities  (including  engineering  support 
from  the  contractor);  lack  of  a  water-activated  parachute 
release  system  (qualification  testing  is  delayed  until 
2013);  incomplete  testing  of  the  escape/ ejection  system; 
low  overall  availability  rates;  and  no  new  information  or 
plans  to  address  deficiencies  in  the  Integrated  Caution  and 
Warning  System. 

-  The  Air  Force  elected  to  begin  the  training  OUE  in  early 
September  2012,  and  concluded  it  in  mid-November  2012. 
The  system  under  test  had  no  combat  capability.  Flight 
training  events  were  limited  to  basic  aircraft  maneuvers 
called  for  in  the  “familiarization”  pilot  transition  syllabus, 
which  is  a  six-flight  module  of  training.  Pilots  were  trained 
in  basic  ground  procedures,  take-off,  approach/landing, 
and  formation  flight.  Radar,  electronic  warfare, 
countermeasures,  and  weapons  capabilities  were  not 
included  in  the  syllabus  as  they  were  either  restricted  from 
being  used  or  were  not  available.  Flight  maneuvering  was 
restricted  to  5.5  g’s,  550  knots,  18  degrees  angle-of-attack, 
and  below  39,000  feet  altitude,  and  was  further  constrained 
by  numerous  aircraft  operating  limitations  that  are  not 
suitable  for  combat.  The  maintenance  environment  and 
support  systems  are  still  immature.  Sortie  generation  was 
dependent  on  contractor  support  personnel,  maintenance 
personnel  had  to  use  workarounds  to  accommodate 
shortfalls  in  ALIS,  and  the  Joint  Technical  Data  was 
incomplete.  DOT&E  will  provide  an  independent  report 
on  the  evaluation  in  early  2013. 

As  of  the  end  of  OUE  in  November,  276  sorties  and  366  hours 
had  been  flown  in  the  F-35A  aircraft  at  Eglin,  with  the  first 
flights  in  March,  and  316  flights  and  410  hours  flown  in  the 
F-35B,  since  starting  in  May. 

-  Aircraft  availability  rates  for  the  F-35  A  varied  from  less 
than  5  percent  to  close  to  60  percent  in  a  given  week  from 
the  first  flights  in  March  through  October,  with  an  average 
availability  of  less  than  35  percent,  meaning  three  of  nine 
aircraft  were  available  on  average  at  any  given  time.  For 
the  F-35B,  availability  rates  varied  monthly  as  well  from 
less  than  5  percent  to  close  to  50  percent,  with  similar 
average  rates  over  the  six  months  of  flying. 

-  Cumulative  air  abort  rates  over  the  same  time  period  were 
also  similar  between  the  two  variants  with  approximately 
five  aborts  per  100  flight  hours  observed  (4.7  for  the  F-35A 
and  5.3  for  the  F-35B).  In  2010,  the  Air  Force  used  air 
abort  rate  as  an  objective  metric  for  assessing  the  maturity 
needed  to  start  flight  training,  with  a  goal  of  1.0  air  abort 
per  100  flight  hours  as  a  threshold  to  start  an  evaluation  of 
the  system’s  readiness  for  training.  Ground  abort  rate  was 
one  ground  abort  in  seven  scheduled  sorties  (0.14)  for  the 
F-35A  and  one  in  eight  (0.13)  for  the  F-35B. 

The  center  conducted  maintenance  training  for  experienced 
maintenance  personnel  for  both  the  F-35A  and  F-35B 
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during  2012.  As  of  the  end  of  November,  542  personnel  had 
completed  training  in  one  or  more  of  the  maintenance  courses. 
Graduates  from  the  maintenance  courses  at  the  training  center 
will  support  initial  service  bed  down  and  training  locations. 

Air  System-Ship  Integration  and  Ship  Suitability  Testing 

F-35B 

•  The  Program  Office  continued  planning  efforts  to  support  the 
next  F-35B  developmental  testing  deployment  to  USS  Wasp 
in  August  2013.  Through  the  middle  of  November,  the  test 
team  had  accomplished  79  vertical  landings  in  2012  (358 
total  to  date)  and  212  short  takeoffs  (631  to  date).  Control 
law  changes  were  made  to  the  vehicle  system  software  as  a 
result  of  flying  qualities  observed  during  the  first  deployment 
to  USS  Wasp  in  2011.  Regression  testing  of  the  control  law 
changes  was  accomplished  in  2012. 

•  Discoveries  affecting  F-35B  operations  on  L-class  ships 
include: 

-  Assessment  of  ship  capabilities  were  inconclusive  in 
determining  whether  there  would  be  adequate  storage 
requirements  for  lithium  battery  chargers  and  spares,  gun 
pods,  and  the  ejection  seat  carts  as  some  of  the  support 
equipment  and  spares  from  legacy  systems  may  no  longer 
be  required.  Additional  data  are  required  to  determine  a 
path  forward. 

-  Propulsion  system  module  containers  do  not  meet  all 
shipboard  requirements.  Due  to  the  fragility  of  certain 
propulsion  system  components,  there  is  significant  risk 
to  engines  during  transport  to  and  from  ships,  using  the 
current  containers.  The  Program  Office  is  coordinating  a 
propulsion  system  fragility  analysis  which  is  expected  to 
lead  to  a  container  redesign. 

-  Concept  of  operations  for  managing  and  using  the 
classified  materials  area  remains  to  be  resolved. 

F-35C 

•  A  redesign  of  the  arresting  hook  system  for  the  F-35C 
to  correct  the  inability  to  consistently  catch  cables  and 
compensate  for  greater  than  predicted  loads  took  place  in 
2012.  The  redesign  includes  modified  hook  point  shape  to 
catch  the  wire,  one-inch  longer  shank  to  improve  point  of 
entry,  addition  of  damper  for  end-of-stroke  loads,  increased 
size  of  upswing  damper  and  impact  plate,  addition  of 
end-of-stroke  snubber.  In  2012,  the  following  occurred: 

-  Initial  loads  and  sizing  study  completed  showed  higher 
than  predicted  loads,  impacting  the  upper  portion  of  the 
arresting  hook  system  (referred  to  as  the  “Y  frame,”  where 
loads  are  translated  from  the  hook  point  to  the  aircraft)  and 
hold  down  damper  (January  2012) 

-  Risk  reduction  activities,  including  cable  rollover 
dynamics  testing  at  Patuxent  River  (March  2012),  deck 
obstruction  loads  tests  at  Lakehurst  (April  2012) 

-  Flight  tests  with  CF-3  using  new  hook  point  and  new  hold 
down  damper  design  at  Lakehurst  (August  2012) 

-  72  of  72  successful  roll-in  tests  with  MK-7  and  E-28  gear 

-  5  of  8  successful  fly-in  tests;  3  of  8  bolters  (missed  wire) 


-  Preliminary  design  review  of  updated  design  completed 
(August  15,  2012) 

•  Analysis  by  Service  and  program  ship  integration  teams 
identified  several  aircraft-ship  interface  problems  for 
resolution. 

-  Deficient  capability  to  transfer  recorded  mission  data  to 
ship  intelligence  functions  for  analysis,  in  particular  video 
data  recorded  by  the  JSF. 

-  Ships  are  unable  to  receive  and  display  imagery 
transmitted  via  Link  16  datalink  by  JSF  (or  other  aircraft). 

-  The  design  of  the  JSF  Prognostic  Health  Maintenance 
downlink  is  incomplete,  creating  concerns  for  sufficient 
interfaces  with  ship  systems  and  Information  Assurance. 

Live  Fire  Test  and  Evaluation 

System-Level  Test  Series 

•  The  program  completed  two  of  the  eight  system-level  test 
series.  The  first,  LF-19D  Flight-Critical  System-Level  test 
series,  was  conducted  on  the  first  F-35A  flight  test  aircraft  to 
assess  the  ballistic  tolerance  of  the  flight  control  system  and 
its  supporting  systems  (e.g.,  power  thermal  management, 
vehicle  management,  and  electrical  power  systems). 

•  This  test  series  targeted  components  of  the  redundant  vehicle 
management  and  electrical  power  systems,  demonstrating 
their  ability  to  automatically  reconfigure  after  damage,  and  to 
continue  to  operate  with  no  obvious  effect  on  the  ability  of  the 
aircraft  to  remain  in  controlled  flight. 

•  The  Live  Fire  Test  team  is  assessing  the  aircraft  vulnerability 
damage  thresholds  and  whether  testing  properly  explored  the 
intended  ballistic  damage  modes  (e.g.,  interference  or  arcing 
between  270  Volt,  28  Volt,  and  signal  lines;  loss  of  flight 
actuator  stiffness;  and/or  impact  to  singularly  vulnerable 
components  such  as  the  flight  actuator  ram  cylinder). 

•  One  test  in  this  series,  LF-19D-27,  demonstrated  aircraft 
vulnerabilities  to  fires  associated  with  leaks  from  the 
PAO  system.  The  aircraft  uses  flammable  PAO  in  the 
avionics  coolant  system,  which  has  a  large  footprint  on 
the  F-35.  The  threat  in  this  ballistic  test  ruptured  the  PAO 
pressure  line  in  the  area  just  below  the  cockpit,  causing  a 
sustained  PAO-based  fire  with  a  leak  rate  of  2.2  gallons  per 
minute  (gpm). 

-  The  program  assessed  that  a  similar  event  in  flight  would 
likely  cause  an  immediate  incapacitation  and  loss  of  the 
pilot  and  aircraft.  The  test  article,  like  the  production 
design,  lacks  a  PAO  shutoff  system  to  mitigate  this 
vulnerability. 

-  In  2008,  the  JSF  Executive  Steering  Board  (JESB)  directed 
the  removal  of  PAO  shutoff  valves  from  the  F-35  design  to 
reduce  the  aircraft  weight  by  2  pounds.  Given  the  damage 
observed  in  this  test,  the  JESB  directed  the  program  to 
re-evaluate  installing  a  PAO  shutoff  system  through  its 
engineering  process  based  on  a  cost/benefit  analysis 

and  the  design  performance  capabilities.  The  ballistic 
test  results  defined  the  significance  of  this  vulnerability. 
However,  the  test  also  showed  that  a  shutoff  system  needs 
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to  outperform  other  fielded  systems.  To  be  effective,  it 
must  trigger  on  smaller  leak  rates,  down  to  2  gpm  versus 
the  6  gpm  typical  of  other  aircraft  designs,  without  causing 
excessive  false  alarms. 

-  The  program  is  currently  working  to  identify  a  low  leak 
rate  technical  solution.  The  Program  Office  will  consider 
operational  feasibility  and  effectiveness  of  the  design, 
along  with  cost,  to  decide  if  PAO  shutoff  valves  will  be 
reinstated  as  part  of  the  production  aircraft  configuration. 

•  Another  test  in  this  series,  LF-19D-16,  identified  the 
vulnerability  associated  with  fuel  fires  from  fueldraulic  system 
leaks.  The  fueldraulic  system  is  a  fuel-based  hydraulic  system 
used  to  control  the  engine  exhaust  nozzle.  It  introduces  a 
significant  amount  of  fuel  plumbing  to  the  aft  end  of  the 
engine  and,  consequently,  an  increased  potential  for  fire. 

-  This  test  confirmed  the  increase  in  vulnerability.  The 
original  aircraft  design  included  flow  fuses,  also  known 
as  excess  flow  check  valves,  to  cutoff  fuel  flow  when  a 
leak  is  sensed  due  to  downstream  fuel  line  damage  or 
failure.  As  a  result  of  the  weight-reduction  initiative, 
the  JESB  directed  removal  of  fueldraulic  fuses  from 
the  production  design  in  2008  to  provide  weight  saving 
of  9  pounds.  Fuses,  however,  were  still  part  of  the 
non-weight-optimized  F-35A  test  article  used  in  this  test. 

-  While  a  ballistic  test  with  fragment  threats  demonstrated 
that  the  fueldraulic  system  poses  a  fire-related 
vulnerability  to  the  F-35,  the  leak  rates  generating  the  fire 
were  insufficient  to  trigger  the  fuses.  Since  the  fuses  did 
not  shut  off  the  flow,  the  result  was  a  sustained  fuel-based 
fire. 

-  The  Program  Office  is  accepting  the  increased 
vulnerability  associated  with  the  fueldraulic  system  and  is 
currently  not  considering  reinstating  the  fueldraulic  fuses 
in  the  production  aircraft  configuration. 

•  A  Computation  of  Vulnerable  Area  Tool  analysis  shows  that 
the  removal  of  the  PAO  coolant  and  fueldraulic  systems 
results  in  a  25  percent  increase  in  aircraft  vulnerability. 

Ballistic  Analysis 

•  The  program  used  a  computational  analysis,  supported  by 
single  fragment  test  data,  to  evaluate  the  vulnerabilities  of  the 
F-35  to  multiple  missile  warhead  fragment  hits  for  several 
encounter  geometries. 

-  Multiple  missile  warhead  fragment  hits  are  more 
combat-representative  and  will  result  in  combined  damage 
effects  that  need  to  be  assessed.  For  example,  aircraft 
may  not  be  lost  due  to  a  fuel  leak  from  a  single  missile 
fragment  impact,  but  combined  leakage  from  multiple 
impacts  could  prevent  the  aircraft  from  returning  to  base. 

-  There  are  potentially  other  such  combined  effects  that 
are  not  known  or  expected  and  that,  due  to  the  analysis 
limitations,  cannot  be  identified.  These  limitations  will 
introduce  a  level  of  uncertainty  in  the  F-35  vulnerability 
assessment. 


•  The  program  used  the  results  of  the  completed  tests  to  assess 
the  effects  of  ballistic  damage  on  the  capability  of  the  aircraft 
to  maintain  controlled  flight. 

-  These  estimates  are  typically  expressed  as  a  function 
of  time  intervals,  i.e.,  0  minutes  (“catastrophic  kill”), 

30  seconds  (“K-kill”),  5  minutes  (“A-kill”),  30  minutes 
(“B-kill”),  etc.;  however,  the  program  categorized  them  in 
terms  that  supported  their  specification  compliance,  i.e., 
“Loss  of  Aircraft”  or  the  ability  to  “Return  to  Forward 
Line  of  Troops  (FLOT).” 

-  These  limited  categories  do  not  provide  detailed  insight 
into  the  vulnerability  of  the  aircraft.  For  example,  with 
a  Return  to  FLOT  criterion  of  55  minutes,  if  the  aircraft 
could  fly  for  45  minutes  it  would  still  be  classified  as 

a  Loss  of  Aircraft  and  no  understanding  is  provided 
concerning  the  aircraft’s  actual  capability  to  maintain 
controlled  flight  for  those  45  minutes.  Such  an  assessment 
does  not  provide  insight  into  the  actual  operational 
survivability  of  the  aircraft  because  it  only  focuses  on  the 
ability  of  the  aircraft  to  fly  for  55  minutes  even  though, 
in  some  instances,  the  pilot  might  need  much  less  time  to 
return  to  friendly  territory. 

STOVL  Propulsion  System  Test  Series 

•  The  program  completed  most  of  the  STOVL  propulsion 
system  test  series.  The  Program  Office  temporarily  suspended 
this  test  series  due  to  budget  constraints  without  notifying 
DOT&E.  The  remaining  lift  fan-to-clutch  drive  shaft  and  lift 
fan  clutch  static  and  dynamic  tests  have  been  postponed  until 
FY13. 

-  The  LFT&E  STOVL  propulsion  system  tests  confirmed 
that  back-ups  to  hydraulic  systems  that  configure  the 
STOVL  propulsion  system  for  its  various  operating  modes 
worked  as  intended. 

-  The  completed  test  events  targeted  the  lift  fan  rotating  and 
stator  components  while  the  fan  was  static.  The  program 
assumed  that  the  lift  fan  would  most  likely  be  hit  while  in 
forward  flight  and  that  hits  during  STOVL  flight  were  less 
likely.  In  most  test  events,  the  system  was  then  run  up  to 
simulate  a  STOVL  landing  sequence. 

-  The  results  indicated  that  test  damage  introduced 
no  measurable  degradation  in  STOVL  propulsion 
performance,  including  cascading  damage  effects,  and 
would  be  undetectable  by  the  system  and  the  pilot. 
Flowever,  due  to  concerns  for  catastrophic  lift  fan  or  drive 
train  damage  that  would  risk  loss  of  the  test  article  for 
subsequent  tests,  this  test  series  did  not  include  dynamic 
tests  to  the  inboard  portion  of  the  lift  fan  blade,  where  the 
cross  section  is  smaller  and  centrifugal  forces  are  higher, 
making  failure  more  likely. 

-  The  engine  manufacturer  is  providing  damage  tolerance 
estimates  for  these  threat-target  conditions,  which  still 
need  to  be  evaluated. 
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Vulnerability  Assessment 

•  The  program  completed  an  intermediate  vulnerability 
assessment  (the  previous  one  was  in  2008)  incorporating 
results  from  ballistic  tests  conducted  to  date,  a 
higher-fidelity  target  model,  and  modified  blast  and  fire 
curves. 

-  The  ORD  requires  an  analysis  of  two  types  of  fragments, 
a  30  mm  high  explosive  incendiary  (HEI)  round  and  a 
Man-Portable  Air  Defense  System  (MANPADS)  missile. 
The  analysis  showed  that  none  of  the  F-35  variants  met 
the  operational  requirements  for  the  HEI  threat.  The 
analysis  also  showed  that  the  F-35  A  and  F-35C  have 
shortfalls  to  the  two  fragment  threats.  The  F-35B  variant 
is  more  resistant  to  these  two  threats,  primarily  due  to 
less  fuel  carried  and  some  additional  shielding  provided 
by  the  lift  fan. 

-  Reinstatement  of  the  dry  bay  fire  extinguishing  system, 
in  combination  with  the  PAO  shutoff  valve,  and  the 
fueldraulic  fuses  could  make  all  F-35  variants  compliant 
for  all  four  specified  ballistic  threats,  as  currently  defined 
in  the  ORD. 

OBIGGS  Redesign 

•  The  program  is  redesigning  OBIGGS  to  address  deficiencies 
identified  in  earlier  fuel  system  simulator  test  series 
(LF-09B)  to  meet  the  vulnerability  requirements  during  all 
critical  segments  of  a  combat  mission  and  to  provide  an 
inert  tank  atmosphere  for  internal  lightning  protection. 

•  The  program  reported  several  design  changes  during  the 
Phase  II  Critical  Design  Review  to: 

-  Fix  the  vent-in-during-dive  problem,  wherein  fresh 
oxygen-laden  air  is  drawn  into  the  fuel  tanks  in  a  dive 

-  More  uniformly  distribute  the  nitrogen  enriched  air 
(NEA)  throughout  the  fuel  tanks 

-  Ensure  NEA  quality 

-  Inform  the  pilot  when  the  system  is  not  inerting  the 
ullage 

•  The  program  will  conduct  verification  and  certification 
testing  and  analyses  to  confirm  the  performance  of  the  new 
OBIGGS  design  on  all  three  aircraft  variants.  These  tests 
are  expected  to  begin  in  FY13. 

•  Additionally,  the  current  fuel  tank  venting  design  is 
inadequate  to  vent  the  tanks  during  a  rapid  descent.  As  a 
result  of  the  related  OBIGGS  and  tank  venting  deficiencies, 
flight  operations  are  currently  not  permitted  within  25  miles 
of  known  lightning  conditions.  Moreover,  below  20,000 
feet  altitude,  descent  rate  is  restricted  to  6,000  feet/minute. 
Dive  rates  can  be  increased  to  up  to  50,000  feet/minute  but 
only  if  the  maneuver  includes  4  minutes  of  level  flight  for 
fuel  tank  pressurization  purposes.  Neither  restriction  is 
acceptable  for  combat  or  combat  training. 

Chemical/Biological  Survivability 

•  The  F-35  Chemical  Biological  Warfare  Survivability 
Integrated  Product  Team  built  and  demonstrated  a  prototype 
full-scale  shelter-liner  for  chemical/biological  containment. 


The  demonstration  did  not  evaluate  effectiveness,  and  the 
program  determined  the  design  was  too  complex  for  field 
use. 

•  The  team  is  working  on  a  lighter,  more  robust  and  less 
complex  redesign.  The  integration  of  the  new  shelter-liner 
with  the  chemical  and  biological  agent  decontamination 
support  system  is  ongoing  with  a  full-up  demonstration  test 
planned  for  FY14. 

Issues  Affecting  Operational  Suitability 

•  Overall  suitability  performance  demonstrates  the  lack  of 
maturity  in  the  F-35  as  a  system  in  developmental  testing 
and  as  a  fielded  system  at  the  training  center. 

•  Reliability  requirements  are  identified  for  system  maturity 
(50,000  fleet  hours),  but  the  program  predicts  a  target  at 
each  stage  of  development  that  projects  growth  toward  the 
maturity  requirement. 

-  Analysis  of  data  through  May  2012  shows  that  flight 
test  and  Lots  1  through  3  aircraft  demonstrated  lower 
reliability  than  those  predictions.  Demonstrated  Mean 
Flight  Hours  Between  Critical  Failure  for  the  F-35A  was 
5.95  hours,  for  the  F-35B  was  4.16  hours,  and  for  the 
F-35C  was  6.71  hours,  which  are  60,  70,  and  84  percent 
of  the  level  predicted  by  the  program  for  this  point  in 
development  of  each  variant,  respectively. 

-  Although  reliability  results  appear  to  indicate 
improvement  over  those  reported  in  last  year’s  report 
(2.65  for  F-35A,  2.05  for  F-35B,  and  2.06  for  F-35C, 
reflecting  data  through  September  2011),  too  few  flight 
hours  have  accrued  (approximately  1.5  percent  of  the 
flight  hours  required  to  achieve  reliability  maturity)  for 
these  results  to  be  predictive,  and  although  they  are  based 
on  a  rolling  three-month  measure  of  reliability,  have 
shown  great  variation  between  measurement  periods. 

•  In  2012,  the  program  updated  the  reliability  growth  plan  for 
the  first  time  since  2006.  Significant  contributors  to  low 
reliability  by  variant  are: 

-  F-35A  -  power  and  thermal  management  system,  CNI, 
lights,  fuel  system,  landing  gear,  fire  control  and  stores, 
integrated  air  vehicle  architecture,  and  electrical  power 
system 

-  F-35B  -  electrical  power  system,  power  and  thermal 
management  system,  integrated  air  vehicle  architecture 
(which  includes  the  Integrated  Core  Processing  system 
and  the  cockpit  displays  including  the  HMDS),  access 
doors  and  covers,  landing  gear,  oxygen  system, 
stabilizers,  lift  fan  system,  crew  escape  and  safety,  and 
flight  control  system 

-  F-35C  -  engine  controls,  power  and  thermal  management 
system,  electrical  power  system,  landing  gear,  and 
integrated  air  vehicle  architecture 

•  The  amount  of  time  spent  on  maintenance,  or  measures 
of  maintainability,  of  flight  test  and  Lots  2  and  3  aircraft 
exceeds  that  required  for  mature  aircraft. 
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-  Mean  corrective  maintenance  time  for  critical  failures  by 
variant  are: 

■  F-35A  -  9.3  hours  (233  percent  of  the  requirement  of 
4.0  hours) 

■  F-35B  -  8.0  hours  (178  percent  of  the  requirement  of 

4.5  hours) 

■  F-35C  -  6.6  hours  (165  percent  of  the  requirement  of 
4.0  hours) 

-  Mean  times  to  repair  by  variant  are: 

■  F-35A  -  4.2  hours  (168  percent  of  the  requirement  of 

2.5  hours) 

■  F-35B  -  5.3  hours  (177  percent  of  the  requirement  of 
3.0  hours) 

■  F-35C  -  4.0  hours  (160  percent  of  the  requirement  of 

2.5  hours) 

-  Maintainability  of  the  system  hinges  on  improvements  and 
maturation  of  Joint  Technical  Data  (JTD),  and  the  ALIS 
functions  that  facilitate  flight  line  maintenance. 

•  The  program  is  developing  and  fielding  the  ALIS  in 
incremental  capabilities,  similar  to  the  mission  systems 
capability  in  the  air  vehicle.  It  is  immature  and  behind 
schedule,  which  has  had  an  adverse  impact  on  maintainability, 
and  delays  delivery  of  aircraft. 

-  ALIS  102  is  a  limited  capability  and  is  the  version  fielded 
only  at  the  Eglin  training  center.  It  was  required  for 
receiving  and  operating  the  early  Lot  2  and  Lot  3  aircraft, 
as  well  as  for  conducting  initial  aircrew  and  maintenance 
training.  This  version  of  ALIS  operates  with  independent 
subsystems  and  requires  multiple  workarounds  to  support 
sortie  generation  and  maintenance  activities. 

-  ALIS  103  is  intended  to  provide  the  initial  integration 
of  ALIS  subsystems.  The  program  intended  to  make  it 
available  for  the  fielding  of  Lot  3  and  Lot  4  production 
aircraft  at  new  operating  locations  in  2012:  Edwards 
AFB,  California,  and  Yuma  Marine  Corps  Air  Station 
(MCAS),  Arizona.  The  program  discovered  problems 
with  ALIS  security  in  February  2012,  which  in  turn 
delayed  the  delivery  of  Lot  3  and  Lot  4  aircraft  from  July 
to  late  in  2012.  A  formal  evaluation  of  ALIS  103  was 
delayed  until  September  2012,  and  was  completed  in 
October.  The  first  F-35B  was  delivered  to  Yuma  MCAS 
on  November  16,  2012,  and  the  first  F-35A  to  Edwards 
AFB  is  delayed  to  December  2012.  These  aircraft  were 
ready  for  delivery  as  early  as  July  2012.  A  version  of 
ALIS  103  has  been  fielded  at  Yuma  MCAS  for  use  with 
ground  operations  of  the  three  Lot  4  F-35Bs  delivered  in 
November.  Flight  operations  at  Yuma  are  expected  to  start 
in  early  2013.  Similarly,  ALIS  103  has  been  fielded  at 
Edwards  AFB  and  is  expected  to  provide  support  delivery 
of  aircraft  and  flight  operations  in  early  2013. 

-  Future  versions  of  ALIS  will  complete  the  integration  of 
subsystems.  In  2012,  the  program  made  limited  progress 
toward  the  development  of  a  deployable  unit-level  version 
of  ALIS  by  demonstrating  only  half  of  the  unclassified 
functionality  on  representative  hardware.  The  deployable 


version  will  weigh  approximately  700  pounds  less  than 
the  existing  2,466-pound  system,  and  will  be  modular  to 
enable  transportation.  Funding  for  development  is  being 
secured  by  the  Program  Office. 

•  The  program  continued  the  process  of  verifying  JTD,  the  set  of 
procedures  used  to  operate  and  maintain  the  aircraft. 

-  As  of  the  end  of  September  2012,  the  program  had  verified 
38  percent  of  the  technical  data  modules  (6,879  out  of  an 
estimated  17,  922),  which  is  close  to  the  planned  schedule. 
The  program  plans  to  have  approximately  11,600  (65 
percent)  of  all  the  modules  verified  by  the  end  of  FY13. 

-  Although  the  program  has  improved  plans  and  dedicated 
effort  for  verifying  and  fielding  JTD,  the  lack  of  JTD 
causes  delays  in  maintenance  actions  that  consequently 
affect  the  availability  of  aircraft. 

•  Data  Quality  and  Integration  Management  (DQIM)  are 
essential  parts  of  the  overall  Autonomic  Logistics  Global 
Sustainment  process  for  the  F-35.  Experiences  with  early 
production  aircraft  indicate  an  immature  database  that  contains 
missing  or  incorrect  part  numbers,  serial  numbers,  and  missing 
scheduling  rules  for  inspections.  Effective  data  quality  and 
integration  management  require  that  part  numbers,  serial 
numbers,  and  inspection  requirements  for  each  aircraft  be 
loaded  into  ALIS  for  mission  debrief  or  maintenance  actions  to 
occur. 

Progress  in  Plans  for  Modification  of  LRIP  Aircraft 

•  The  program  and  Services  continued  planning  for 
modifications  of  early  LRIP  aircraft  to  attain  planned  service 
life  and  the  final  SDD  Block  3  capability. 

-  In  January,  the  aircraft  assembly  plant  received  the  first 
production  wing  parts,  which  the  program  redesigned  as 
a  result  of  life  limits  imposed  by  structural  analyses.  The 
assembly  plant  received  the  first  F-35  A  forward  root  rib 
in  January  for  in-line  production  of  AF-31,  the  first  Lot  5 
F-35A  aircraft,  which  is  scheduled  to  deliver  in  2013. 

-  The  operational  test  agencies  worked  with  the  Services 
and  the  Program  Office  to  identify  modifications  required. 
Due  to  the  extension  of  the  program,  which  resulted  in 
very  early  procurement  (relative  to  the  end  of  SDD)  of 
the  aircraft  planned  for  IOT&E,  there  is  high  risk  that  the 
Service  plans  for  updating  the  aircraft  intended  for  IOT&E 
will  not  be  production-representative.  Activities  to  study 
the  depth  of  the  problem  occurred  in  2012;  however,  a 
comprehensive,  funded  plan  that  assures  a  production- 
representative  set  of  aircraft  for  OT&E  is  not  yet  available. 
This  is  a  significant  and  fundamental  risk  to  an  adequate 
IOT&E. 

-  The  first  set  of  depot-level  modifications  for  the  F-35  A 
aircraft  are  scheduled  to  begin  at  Hill  AFB  in  early  2014. 
Initial  F-35B  modifications  will  be  completed  at  the  initial 
operating  base  at  Yuma  MCAS,  Arizona.  Modification  of 
the  Auxiliary  Air  Inlet  Door,  which  is  required  for  vertical 
landings,  has  begun  on  the  first  F-35B  delivered  to  Yuma  in 
November. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  program  and 
Services  are  satisfactorily  addressing  four  of  seven  previous 
recommendations.  The  remaining  three  recommendations 
concerning  use  of  objective  criteria  for  evaluating  flight  test 
progress,  integrating  flight  test  of  an  operational  mission  data 
load,  restoring  shut-off  valves,  and  redesigning  the  OBIGGS 
are  outstanding. 

•  FY12  Recommendations.  The  program  should: 

1.  Make  the  corrections  to  Revision  4  of  the  JSF  TEMP, 
as  described  by  DOT&E  September  2012  memorandum 
disapproving  the  TEMP 

-  Include  the  electronic  warfare  test  annex  that  specifically 
required  operationally-realistic  threats 

-  Include  adequate  criteria  for  entering  the  final  preparation 
period  prior  to  IOT&E 

-  Schedule  the  start  of  the  final  preparation  period  prior 
to  IOT&E  to  begin  no  earlier  than  the  Operational 
Test  Readiness  Review,  approximately  90  days  prior 
to  the  end  of  the  air-worthiness  certification  phase  of 
development 

2.  Conduct  dedicated  ALIS  end-to-end  developmental 
testing  of  each  incremental  ALIS  version  that  supports  the 
production  aircraft. 

3.  Assure  modification  and  retrofit  plans  for  OT  aircraft  make 
these  aircraft  fully  production-representative. 

4.  Ensure  the  contractor  is  meeting  VSim  requirements  for 
operational  testing  and  is  addressing  data  requirements  to 
support  the  validation,  verification,  and  accreditation  during 
developmental  testing. 


5.  Assure  the  schedules  of  record  for  weapons  integration, 
VSim,  and  mission  data  load  production/verification  are 
consistent  with  the  Integrated  Master  Schedule. 

6.  Continue  with  the  OBIGGS  redesign  efforts  to  ensure  the 
system  has  the  capability  to  protect  the  aircraft  from  threat 
and  lightning  induced  fuel  tank  explosions  while  on  the 
ground  and  during  all  phases  of  a  combat  mission  without 
compromised  maneuver  limits. 

7.  Continue  the  PAO  system  redesign  efforts  and  reinstall  a 
PAO  shutoff  valve  to  protect  the  aircraft  from  PAO-based 
fires. 

8.  Reconsider  the  removal  of  the  fueldraulic  fuses.  The 
program  should  design  and  reinstate  an  effective  engine 
fueldraulic  shutoff  system  to  protect  the  aircraft  from 
fuel-induced  fires. 

9.  Reconsider  the  removal  of  a  dry  bay  fire  extinguisher 
system  from  other  than  the  Integrated  Power  Package 
dry  bay.  Prior  F-35  Live  Fire  testing  showed  that  the  fire 
suppression  system  could  be  designed  to  successfully 
extinguish  fires  from  the  most  severe  ballistic  threats. 

10.  Provide  a  higher-resolution  estimate  on  how  long  the 
aircraft  could  continue  to  maintain  controlled  flight  after  a 
ballistic  event.  Remaining  flight  time,  expressed  in  smaller 
time  intervals  (e.g.,  30  seconds,  5  minutes,  30  minutes, 
etc.)  is  a  more  informative  metric  than  the  current  “Loss  of 
Aircraft”  or  “Return  to  FLOT”  metric. 
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Global  Command  and  Control  System  -  Joint  (GCCS-J) 


Executive  Summary 

•  Defense  Information  Systems  Agency  (DISA)  development 
focused  on  implementing  high  priority  capability 
enhancements,  software  corrections,  and  infrastructure 
improvements  to  both  Global  Command  and  Control  System 
-  Joint  (GCCS-J)  Global  and  Joint  Operation  Planning  and 
Execution  System  (JOPES). 

GCCS-J  Global 

•  DISA  developed  GCCS-J  Global  v4.2.0.9  (Global 
v4.2.0.9)  to  provide  operational  enhancements,  remediate 
security  vulnerabilities,  and  correct  Common  Operational 
Picture  (COP)  and  Integrated  Imagery  and  Intelligence 
(13)  application  deficiencies  discovered  while  testing 
and  operating  previous  Global  releases.  DISA  expended 
significant  effort  to  ensure  Global  v4.2.0.9  would  be 
acceptable  for  fielding  into  the  Air  Operations  Centers 
(AOCs)  that  support  the  Joint  Force  Air  Component 
Commanders.  Global  v4.2.0.9  testing  occurred  at  both  the 
Combatant  Command  (CCMD)  and  AOC  levels. 

•  CCMD-level  testing: 

-  The  Joint  Interoperability  Test  Command  (J1TC) 
conducted  CCMD-level  operational  testing  in 
December  2011,  revealing  substantially  fewer  defects 
than  OT&E  conducted  during  the  past  two  years. 
However,  due  to  known  and  encountered  test  limitations, 
operational  testing  was  insufficient  to  determine 
operational  effectiveness  and  suitability  for  both  the 
CCMD  and  higher-echelon  environments  in  which 
Global  v4.2.0.9  will  be  used. 

-  Regression  testing  of  Global  v4.2.0.9  Updates  1  and  2 
conducted  by  DISA  in  2012  showed  the  release  was 
acceptable  at  CCMD  and  higher  echelons.  Four 
Category  1  deficiencies  and  15  Category  II  deficiencies 
remain.  Testing  in  the  more  demanding  AOC 
environment  is  required  to  fully  evaluate  Global  v4.2.0.9 
corrections. 

•  AOC-level  testing: 

-  DOT &E  used  data  collected  during  AOC-level 
developmental  and  operational  testing,  which 
implemented  Global  4.2.0. 9  in  a  more  complex 
configuration,  to  assess  Global  v4.2.0.9  performance  at 
lower  echelons. 

-  The  Air  Force  conducted  developmental  testing  at 
Langley  AFB,  Virginia,  of  Air  Operations  Center  - 
Weapon  System  (AOC-WS)  10.1  Recurring  Event 
(RE)ll,  which  used  Global  v4.2.0.9  as  a  significant 
portion  of  RE11. 

-  The  605th  Test  and  Evaluation  Squadron  (TES) 
conducted  a  Force  Development  Evaluation  (FDE)  of 
RE11  with  Global  v4.2.0.9  Update  1,  which  revealed 
that  most  critical  deficiencies  had  been  fixed,  but  faster 
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servers  would  be  required  to  process  the  data  demands  of 
the  AOC  environment.  Global  v4.2.0.9  Update  1  testing 
occurred  on  upgraded  servers,  and  showed  a  four-fold 
improvement  in  processing  speed,  satisfying  AOC  data 
demands. 

-  605th  TES  integrated  d  c  ve  I  o  p  m  e  n  ta  I  o  p  era  t  i  o  n  a  I  testing 
of  RE  12  with  Global  v4.2.0.9  Update  2  in  December 
2012  is  expected  to  address  the  remaining  4  Category  1 
and  15  Category  II  deficiencies  of  Global  v4.2.0.9  within 
the  AOC  environment. 

GCCS-J  JOPES 

•  DISA  developed  GCCS-J  JOPES  v4.2.0.2  (JOPES  v4.2.0.2) 
to  provide  the  initial  framework  for  the  Transportation 
Tracking  Account  Number  (TTAN),  to  upgrade  JOPES 
Data  Network  Services  (JDNETS)  software  to  reduce 
dependency  on  the  Deliberate  and  Crisis  Action  Planning 
and  Execution  Segment  (DC APES)  and  support  enhanced 
system  monitoring,  to  resolve  existing  problem  reports,  and 
to  upgrade  commercial  off-the-shelf  (COTS)  software. 

•  JITC  conducted  low-risk  operational  testing  in  February 
and  March  2012  of  JOPES  v4.2.0.2  at  five  CCMD  sites, 
three  Service  sites,  the  Joint  Staff  Support  Center,  and  at 
the  DISA  Headquarters  facility  laboratories  at  Fort  George 
G.  Meade,  Maryland.  JOPES  operational  users  validated 
the  JOPES  v4.2.0.2  capabilities  during  the  execution  of  an 
end-to-end  Time-Phased  Force  Deployment  Data  scenario. 

-  The  scenario  demonstrated  the  ability  to  identify  high- 
level  mission  requirements,  source  operational  forces 
and  materials,  verify  force  and  material  availability  and 
mission  readiness,  and  schedule  transportation  to  move 
those  forces  and  materials. 
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-  Decoupling  of  database  exchanges  between  DCAPES 
and  JOPES  allowed  each  program  to  upgrade  without 
affecting  the  other. 

-  Enhanced  system  monitoring  allowed  JOPES 
system  administrators  to  better  track  JOPES  system 
performance,  such  as  transaction  queues  and  query 
backlogs,  across  all  four  enclaves  and  deployable 
servers. 

System 

•  GCCS-J  is  a  command  and  control  system  utilizing 
communications,  computers,  and  intelligence  capabilities. 

The  system  consists  of  hardware,  software  (COTS  and 
government  off-the-shelf),  procedures,  standards,  and 
interfaces  that  provide  an  integrated  near  real-time  picture  of 
the  battlespace  necessary  to  conduct  joint  and  multi-national 
operations.  GCCS-J  consists  of  a  client/server  architecture 
using  open  systems  standards,  government-developed  military 
planning  software,  and  an  increasing  use  of  World  Wide  Web 
technology. 

•  GCCS-J  consists  of  two  main  components: 

-  Global  v4.2.0.9  (Force  Protection,  Situational  Awareness, 
Intelligence  applications) 

-  JOPES  v4.2.0.2  (Force  Employment,  Projection,  Planning, 
and  Deployment/Redeployment  applications) 

Mission 

•  Joint  Commanders  utilize  the  GCCS-J  to  accomplish 
command  and  control. 

GCCS-J  Global 

•  Commanders  use  GCCS-J  Global: 

-  To  link  the  National  Command  Authority  to  the  Joint 
Task  Force,  Component  Commanders,  and  Service 
unique  systems  at  lower  levels  of  command 


-  To  process,  correlate,  and  display  geographic  track 
information  integrated  with  available  intelligence  and 
environmental  information  to  provide  the  user  a  fused 
battlespace  picture 

-  To  provide  13  capabilities 

-  To  provide  a  missile  warning  and  tracking  capability 

•  The  AOC  uses  GCCS-J  Global: 

-  To  build  the  air  picture  portion  of  the  COP  and  maintain 
its  accuracy 

-  To  correlate  or  merge  raw  track  data  from  multiple 
sources 

-  To  associate  raw  Electronics  Intelligence  data  with  track 
data 

-  To  perform  targeting  operations 

GCCS-J  JOPES 

•  Commanders  use  GCCS-J  JOPES: 

-  To  translate  policy  decisions  into  operation  plans 
(OPLANs)  to  meet  U.S.  requirements  for  the 
employment  of  military  forces 

-  To  support  force  deployment,  redeployment,  retrograde, 
and  reposturing 

-  To  conduct  contingency  and  crisis  action  planning 

Major  Contractors 

•  Government  Integrator:  DISA 

•  Software  Developers: 

-  Northrop  Grumman  -  Arlington,  Virginia 

-  SAIC  -  Arlington,  Virginia 

-  Pragmatics  -  Arlington,  Virginia 


Activity 

GCCS-J  Global 

•  DISA  developed  Global  v4.2.0.9  to  provide  operational 
enhancements,  remediate  security  vulnerabilities,  and 
correct  COP  and  13  application  deficiencies  discovered 
while  testing  and  operating  previous  Global  releases.  DISA 
expended  significant  effort  to  ensure  Global  v4.2.0.9  would 
be  acceptable  for  fielding  into  the  AOCs  that  support  the 
Joint  Force  Air  Component  Commanders. 

•  Global  4. 2. 0.9  is  a  critical  upgrade  urgently  needed  by 
the  AOC-WS  program  and  the  operational  AOCs.  The 
AOC-WS  baseline  uses  legacy  Global  v4.0.2  and  cannot 
employ  critical  13  capabilities  due  to  outstanding  Category 

I  problems  when  used  in  the  AOC  operational  environment. 
Global  v4.2.0.9  testing  occurred  at  both  the  CCMD  and 
AOC  levels. 

•  CCMD-level  testing: 

-  JITC  conducted  operational  testing  of  Global  v4.2.0.9  at 
U.S.  European  Command  and  U.S.  Southern  Command 


in  December  2011  in  accordance  with  a  DOT&E- 
approved  test  plan. 

-  DISA  conducted  a  regression  test  of  Global  v4.2.0.9 
Update  1  in  January  and  February  2012,  after  correction 
of  deficiencies. 

-  DISA  conducted  a  regression  test  of  Global  v4.2.0.9 
Update  2  in  June  2012  after  making  corrections  to 
additional  deficiencies  identified  during  Update  1  and 
RE  11  testing. 

•  AOC-level  testing: 

-  The  Air  Force  performed  developmental  testing  of  RE  1 1 
in  January  2012  at  Langley  AFB,  Virginia.  A  significant 
portion  of  this  testing  involved  additional  testing  of  the 
Global  v4.2.0.9  baseline  as  it  was  approved  by  DISA  for 
fielding  within  an  AOC  environment. 

-  The  605  th  TES  performed  an  FDE  of  RE  11,  to 
include  the  corrections  in  Global  v4.2.0.9  Update  1, 
at  Langley  AFB  in  March  2012.  Additional  testing  of 
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Global  v4.2.0.9  Update  1,  modified  to  work  on  newer, 
more  powerful  servers,  occurred  at  Langley  AFB  in 
May  2012.  During  RE11,  the  605th  TES  implemented 
Global  v4.2.0.9  in  a  more  complex  configuration.  This 
configuration  has  more  live  interfaces  and  the  ability 
to  load  up  to  30,000  active  tracks  into  the  battlespace 
picture,  which  was  significantly  more  demanding  than 
previous  operational  testing. 

-  The  605th  TES  plans  to  test  Global  v4.2.0.9  Update  2  in 
an  AOC  environment  in  December  2012. 

GCCS-J  JOPES 

•  DISA  developed  JOPES  v4.2.0.2  to  provide  the  initial 
framework  for  TTAN,  to  upgrade  JDNETS  software  to 
reduce  dependency  on  DCAPES  and  support  enhanced 
monitoring,  to  resolve  existing  problem  reports,  and  to 
upgrade  COTS  software. 

•  JITC,  in  conjunction  with  the  GCCS-J  Program  Office, 
conducted  low-risk  operational  testing  of  JOPES  v4.2.0.2  at 
five  CCMD  sites,  three  Service  sites,  the  Joint  Staff  Support 
Center,  and  at  the  DISA  Headquarters  facility  laboratories 
at  Fort  George  G.  Meade,  Maryland.  Testing  occurred  in 
February  and  March  2012. 

GCCS-J  SORTS 

•  DISA  transitioned  the  GCCS-J  Status  of  Resources  and 
Training  System  (SORTS)  component  to  the  Defense 
Readiness  Reporting  System  Implementation  Office  in  late 
October  2011. 

Assessment 

GCCS-J  Global 

•  DOT&E  approved  the  JITC  plan  for  operationally  testing 
Global  v4.2.0.9  for  fielding  to  CCMD  and  higher-echelon 
locations,  but  not  for  fielding  at  lower-echelon  sites 
such  as  AOCs.  DOT&E  is  using  operational  test  data 
collected  during  RE  11  to  adequately  assess  Global  v4.2.0.9 
performance  at  lower  echelons. 

•  CCMD-level  testing: 

-  Results  of  JITC  operational  testing  demonstrated 
significant  improvement  relative  to  past  testing. 
Substantially  fewer  defects  were  discovered  during  the 
CCMD  testing  than  was  the  case  in  nearly  every  OT&E 
conducted  during  the  past  two  years.  However,  the 
aggregate  of  known  and  encountered  test  limitations 
rendered  this  test  insufficient  to  determine  operational 
effectiveness  and  suitability  for  both  the  CCMD  and 
higher-echelon  environments  in  which  Global  v4.2.0.9 
will  be  used. 

-  Many  Global  v4.2.0.9  interfaces  were  not  available  or 
tested  at  the  two  CCMD  test  locations,  and  stress  levels 
for  the  COP  and  intelligence  portion  of  GCCS-J  are  not 
operationally  representative  for  AOC  usage. 

-  The  CCMD  COP  loading  was  well  below  the  30,000 
and  35,000  active  tracks,  with  missile  events,  expected 
within  the  AOC  environment. 


-  Each  CCMD  location  had  only  three  to  six  intelligence 
users  participating  in  the  test,  whereas  lower-echelon 
locations  may  have  over  100  users. 

-  Regression  testing  in  May  2012  showed  that 
Global  v4.2.0.9  Update  1  was  acceptable  at  CCMD 
and  higher  echelons,  albeit  with  4  Category  I  and  15 
Category  II  deficiencies  requiring  correction  before 
fielding  to  AOCs.  Regression  testing  on  Update  1 
demonstrated  the  ability  of  Global  v4.2.0.9  Update  1  to 
handle  the  more  demanding  track  loads  experienced  in 
the  AOC  environment. 

-  DISA  conducted  regression  testing  of  Global  v4.2.0.9 
Update  2  to  validate  corrections  for  1  of  4  Category  1  and 
10  of  15  Category  II  AOC  deficiencies  identified  during 
Update  1  and  RE  1 1  testing.  However,  testing  in  an  AOC 
environment  is  required  to  fully  evaluate  Global  v4.2.0.9 
corrections. 

•  AOC-level  testing: 

-  DOT&E  used  data  collected  at  Langley  AFB,  Virginia, 
during  developmental  and  operational  testing  of  RE11, 
which  used  Global  v4.2.0.9  as  a  significant  portion 

of  RE1 1,  to  assess  Global  v4.2.0.9  performance  at 
lower  echelons.  RE  11  offered  a  test  environment 
more  operationally  representative  of  AOC  usage  and 
is  expected  to  provide  the  remaining  operational  data 
needed  to  adequately  assess  the  system  once  testing  is 
complete. 

-  Developmental  testing  of  the  RE  11  revealed  several 
critical  deficiencies  with  the  Global  v4.2.0.9  baseline. 
JITC  did  not  find  these  deficiencies  during  the  earlier 
operational  testing  at  the  two  CCMD  sites  due  to  test 
limitations.  AOC  users  applied  more  stress  to  the  Global 
COP  and  13  application  suite  than  was  experienced  at  the 
CCMD  test  sites. 

-  The  FDE  of  RE  11  with  Global  v4.2.0.9  Update  1 
revealed  that  the  most  critical  deficiencies  had  been 
fixed,  but  that  faster  servers  would  be  required  to  process 
the  data  demands  experienced  in  the  AOC  environment. 
Global  v4.2.0.9  Update  1  was  tested  on  upgraded 
servers,  showing  a  four-fold  improvement  in  processing 
speed  and  satisfying  AOC  data  demands. 

-  605th  TES  integrated  developmental/operational  testing 
of  RE  12  with  Global  v4.2.0.9  Update  2  in  December 
2012  is  expected  to  address  the  remaining  4  Category  I 
and  15  Category  II  deficiencies  of  Global  v4.2.0.9  within 
the  AOC  environment. 

GCCS-J  JOPES 

•  JOPES  operational  users  validated  the  JOPES  v4.2.0.2 
capabilities  during  the  execution  of  an  end-to-end 
Time-Phased  Force  Deployment  Data  scenario.  JITC  also 
observed  DISA  testers  perform  functional  testing  of  the 
system  to  verify  that  system  changes  did  not  introduce  new 
errors  to  the  previously  fielded  software. 
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-  The  supported  CCMD  identified  high-level  requirements 
for  forces  and  materials  to  accomplish  a  particular 
OPLAN. 

-  CCMD  users  sourced  operational  forces  and  materials  to 
support  the  OPLAN,  along  with  required  timelines  and 
locations. 

-  CCMD  users  verified  forces  and  material  were  available 
and  mission-ready. 

-  U.S.  Transportation  Command  users  scheduled 
transportation  for  the  operational  forces  and  materials  to 
meet  the  supported  CCMD  requirements. 

•  TTAN  implementation  added  new  data  fields,  providing 
operational  users  with  an  initial  capability  to  track  the 
status  of  individuals  and  pieces  of  equipment  for  improved 
situational  awareness.  Upgrades  to  JDNETS  web  services 
did  not  adversely  affect  users’  ability  to  monitor,  plan, 
and  execute  mobilization,  deployment,  employment,  and 
sustainment  activities  associated  with  joint  operations. 


The  upgrades  to  JDNETS  improved  JOPES  administrator 
system-monitoring  capabilities  with  an  enhanced 
dashboard.  Users  successfully  installed  COTS  software 
upgrades,  following  installation  procedures,  with  no 
discrepancies  noted. 

Recommendations 

•  Status  of  Previous  Recommendations.  DISA  addressed  all 
previous  recommendations. 

•  FY12  Recommendation. 

1.  Operational  testing  of  significant  Global  v4.2.0.9  upgrades 
will  require  testing  by  both  CCMD  and  AOC  communities 
to  successfully  address  the  significant  test  limitations 
encountered  at  the  CCMD  locations.  Operational  testing 
needs  to  include  operationally  representative  stress  levels 
by  users,  significant  COP  data  processing  of  dynamically 
updating  tracks,  and  live  data  flowing  across  all  critical 
interfaces. 
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Joint  Tactical  Radio  System  (JTRS) 
Handheld,  Manpack,  and  Small  Form  Fit  (HMS) 

Manpack  Radio 


Executive  Summary 

•  In  May  2011,  the  Defense  Acquisition  Executive  (DAE) 
approved  a  Milestone  C  Low-Rate  Initial  Production  (LRIP) 
decision  of  100  Manpack  radios. 

•  In  June  2011,  the  Army  conducted  a  Manpack  Limited 
User  Test  (LUT)  as  a  part  of  its  2011  Network  Integration 
Evaluation  (NIE)  11.2.  The  Manpack  radio  demonstrated 
poor  reliability,  transmission  range,  and  voice  quality  that 
restricted  the  unit’s  ability  to  accomplish  its  mission.  These 
same  problems  were  observed  during  previous  developmental 
testing. 

•  In  May  2012,  the  Deputy  Assistant  Secretary  of  Defense, 
Developmental  Test  and  Evaluation  (DASD  DT&E)  published 
a  Manpack  Assessment  of  Operational  Test  Readiness  (AOTR) 
that  stated  the  radio  was  not  sufficiently  mature  to  enter 

the  planned  Manpack  Multi-Service  Operational  Test  and 
Evaluation  (MOT&E).  DASD  DT&E  recommended  that  the 
Manpack  radio  not  proceed  to  MOT&E  to  allow  for  corrective 
actions  and  additional  developmental  testing. 

•  In  May  2012,  the  Army  Test  and  Evaluation  Command 
(ATEC)  conducted  the  Manpack  radio  MOT&E  as  a  part 
of  its  NIE  12.2.  DOT&E  assessed  the  Manpack  radio  as 
not  operationally  effective  due  to  the  poor  performance  of 
the  Single  Channel  Ground  and  Airborne  Radio  System 
(SINCGARS)  waveform  and  not  operationally  suitable  due 
to  a  failure  to  meet  reliability  or  availability  requirements. 

The  Manpack  radio  AOTR  had  outlined  these  major  MOT&E 
deficiencies  prior  to  operational  test. 

•  In  September  2012,  the  Army  conducted  a  Govermnent 
Development  Test  (GDT)  3  to  demonstrate  improvements  in 
MOT&E  deficiencies  to  support  a  planned  1QFY13  second 
LRIP  decision. 

•  In  October  2012,  the  DAE  approved  a  second  LRIP  decision 
for  3,726  Manpack  radios. 

•  The  Joint  Tactical  Radio  System  (JTRS)  Handheld,  Manpack, 
and  Small  Fonn  Fit  (HMS)  program  is  schedule-driven  and 
has  reduced  developmental  testing  to  support  an  aggressive 
operational  test  schedule.  Operational  testing  continues 

to  reveal  problems  that  developmental  testing  should  have 
identified  and  fixed. 

•  The  Army  continues  preparation  for  a  future  Manpack  radio 
MOT&E-2,  which  will  include  competition  of  Program  of 
Record  and  alternate  vendors. 

System 

•  JTRS  is  a  family  of  software-programmable  and 
hardware-configurable  digital  radios  intended  to  provide 


increased  interoperability,  flexibility,  and  adaptability  to 
support  numerous  tactical  communications  requirements. 

•  The  JTRS  HMS  program  provides  handheld  and  two-channel 
Manpack  radios  supporting  Army,  Marine  Corps,  Navy,  and 
Air  Force  operations.  The  program  develops  Small  Form  Fit 
(SFF)  radio  configurations  that  include  the  stand-alone  Army 
Rifleman  Radio  and  embedded  SFF  variants  that  serve  in 
Army  host  platforms  such  as  the  SFF-B  (Shadow  Unmanned 
Aerial  Vehicle),  SFF-B  (V)l  (Nett  Warrior),  and  the  SFF-D 
(Small  Unmanned  Ground  Vehicle). 

•  The  program  strategy  has  two  phases  of  HMS  production.  The 
JTRS  HMS  program  developed  the  Rifleman  Radio  as  part 

of  its  Phase  1  effort  to  provide  software  programmable  radios 
with  National  Security  Agency  (NSA)  Type  2  encryption 
of  unclassified  information  that  could  operate  a  networking 
waveform.  Phase  2  consists  of  developing  the  Manpack  radio 
to  provide  software  programmable  radios  with  NSA  Type  1 
encryption  of  classified  information. 

•  The  Manpack  radio  is  a  two-channel  radio  with  military  GPS 
that: 

-  Is  capable  of  operating  at  various  transmission  frequencies 
using  the  Soldier  Radio  Waveform  (SRW),  the  SINCGARS 
waveform,  and  current  military  satellite  communications 
waveforms.  The  JTRS  HMS  program  intends  to  host  the 
Mobile  User  Objective  Satellite  waveform  on  the  Manpack 
radio  as  an  objective  capability. 

-  Operates  up  to  20  watts  at  maximum  power  output. 

-  Allows  Soldiers  to  participate  in  doctrinal  networks  and 
transmit  Position  Location  Information. 
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Mission 

Commanders  from  the  Army,  Marine  Corps,  Navy,  and  Air  Force  in  land  combat  operations  to  support  voice,  video,  and  data 
use  Manpack  radios  to:  across  the  air,  land,  and  sea  battlespace. 

•  Communicate  and  create  networks  to  exchange  voice,  video, 

and  data  using  legacy  waveforms  or  the  SRW  during  all  Major  Contractor 

aspects  of  military  operations.  General  Dynamics,  C4  Systems  -  Scottsdale,  Arizona 

•  Integrate  JTRS  SFF  variants  into  host  platforms  to  provide 
networked  communications  capabilities  for  users  engaged 


Activity 

•  The  Army  conducted  four  developmental  tests  of  the 
Manpack  radio: 

-  Manpack  Customer  Test,  conducted  at  Fort  Benning, 
Georgia,  February  7  -  11,  2011. 

-  Manpack  GDT,  conducted  at  the  Electronic 
Proving  Ground  (EPG),  Fort  Fluachuca,  Arizona, 

April  15  -  22,  2011  (originally  planned  for  45  days). 

-  Manpack  GDT  2,  conducted  at  the  EPG,  Fort  Fluachuca, 
Arizona,  March  2  -  30,  2012. 

-  Manpack  GDT  3,  conducted  at  the  EPG,  Fort  Fluachuca, 
Arizona,  September  19  through  October  3,  2012. 

•  In  May  2011,  the  DAE  approved  a  Milestone  C  LR1P  decision 
to  procure  100  Manpack  radios  of  a  total  acquisition  objective 
of  71,814  radios.  The  Manpack  radio  LR1P  is  intended  to 
support  future  developmental  and  operational  tests. 

•  In  June  2011,  the  Army  conducted  the  Manpack  LUT  as  part 
of  its  N1E  11.2  at  White  Sands  Missile  Range,  New  Mexico. 
The  Army  used  the  LUT  to  assess  the  performance  of  the 
Manpack  radio  under  numerous  mission  scenarios  executed 
by  a  cavalry  troop. 

•  In  March  2012,  the  program  conducted  GDT  2  to  assess  the 
Manpack  radio  and  verify  fixes  of  deficiencies  highlighted 
during  the  20 1 1  LUT. 

•  In  May  2012,  ATEC  conducted  the  Manpack  radio  MOT&E 
as  part  of  the  Army’s  NIE  12.2  at  White  Sands  Missile 
Range,  New  Mexico.  ATEC  tested  the  Manpack  radio  under 
various  missions  conducted  by  a  motorized  infantry  company. 
Soldiers  used  the  Joint  Enterprise  Network  Manager  to  plan, 
load,  and  monitor  the  Manpack  radio  waveforms.  The  test 
was  conducted  in  accordance  with  a  DOT&E-approved  test 
plan. 

•  The  Army  conducted  a  Manpack  radio  GDT  3,  from 
September  through  October  2012.  The  test  was  intended  to 
verify  fixes  to  reliability  and  performance  deficiencies  found 
during  the  MOT&E  and  GDT  2. 

•  On  October  1 1,  2012,  the  DAE  approved  a  second  LR1P  for 
an  additional  3,726  Manpack  radios  to  increase  the  total  LRIP 
procurement  to  5  percent  of  the  total  acquisition  objective. 

•  The  Army  is  developing  a  JTRS  F1MS  Manpack  Radio 
Acquisition  Strategy  Report  and  Test  and  Evaluation  Master 
Plan.  These  documents  are  required  for  future  developmental 
and  operational  testing. 


Assessment 

•  The  Army  reduced  the  first  Manpack  GDT  (April  2011)  from 
a  scheduled  45  days  to  8  days  in  order  to  place  radios  into  the 
NIE  11.2  Manpack  LUT. 

•  Both  the  Manpack  Customer  Test  and  GDT  highlighted 
deficiencies  in  performance  and  poor  reliability.  The  Army 
determined  that  the  Manpack  radio’s  S1NCGARS  waveform 
was  not  ready  for  test  and  therefore  did  not  test  it  during  the 
shortened  GDT. 

•  During  the  NIE  11.2  Manpack  LUT,  the  radio  demonstrated 
the  following: 

-  Ability  to  transmit  and  receive  on  two  channels 

-  Ability  to  distribute  Position  Location  Information 
throughout  the  network 

-  Poor  reliability 

-  Poor  transmission  range  performance  of  the  SRW  and 
S1NCGARS  waveforms  that  constricted  the  operational 
area  of  the  cavalry  troop 

-  Inconsistent  voice  quality 

-  SINCGARS  waveform  did  not  support  unit  operations  and 
was  immature  for  operational  test 

•  The  NIE  1 1.2  Manpack  LUT  reliability  data  collection  was 
not  adequate  and  not  conducted  in  accordance  with  the 
DOT&E-approved  test  plan. 

•  The  Manpack  radio  in  GDT  2  demonstrated  improved 
performance  of  the  SRW,  but  the  performance  of  the 
SINCGARS  waveform  and  reliability  were  poor. 

•  In  May  2012,  DASD  DT&E  published  a  Manpack  radio 
AOTR  that  stated  the  radio  was  not  sufficiently  mature  to  enter 
the  planned  MOT&E  due  to  developmental  test  deficiencies 
that  included  poor  reliability  and  an  immature  SINCGARS 
waveform.  DASD  DT&E  recommended  that  the  Manpack 
radio  not  proceed  to  MOT&E  to  allow  for  corrective  actions 
and  more  developmental  testing. 

•  Based  on  the  NIE  12.2  Manpack  radio  MOT&E,  DOT&E 
made  the  following  assessment: 

-  Not  operationally  effective  due  to  the  poor  voice  quality 
and  limited  range  of  the  SINCGARS  waveform  compared 
to  legacy  SINCGARS  radios.  Since  the  SINCGARS 
performance  was  poor,  the  company  leadership  resorted 
to  using  satellite -based  chat  communications  of  the  Blue 
Force  Tracker. 
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The  SRW  performance  was  good  and  the  Soldiers  were 
able  to  employ  the  Manpack  radio  for  intra-company  voice 
and  data  communications. 

-  Not  operationally  suitable  due  to  a  failure  to  meet 
reliability  and  availability  thresholds. 

-  No  waveform  met  the  Army’s  reliability  requirement.  The 
SRW  was  the  most  used  waveform  and  demonstrated  a 
reliability  of  163  hours  Mean  Time  Between  Essential 
Function  Failure  (MTBEFF)  compared  to  the  radio’s 
revised  requirement  of  477  hours.  This  translates  to 

a  63  percent  chance  of  completing  a  72-hour  mission 
compared  to  a  requirement  of  86  percent. 

-  No  waveform  met  the  availability  requirement.  The  SRW 
achieved  an  operational  availability  of  0.86  compared  to  a 
0.96  requirement. 

-  The  radio’s  design  allowed  Soldiers  to  accidentally  zero 
the  Manpack  radio.  This  action  erases  all  radio  presets  and 
communications  security,  and  requires  20  -  25  minutes  to 
restore  the  Manpack  radio  to  operation. 

-  The  Army’s  integration  of  the  radios  into  Mine  Resistant 
Ambush  Protected  vehicles  was  poor  and  reduced  the 
radio’s  performance.  The  program  did  not  test  vehicle 
integration  in  developmental  testing  prior  to  the  MOT&E. 

•  During  GDT  3,  the  Manpack  radio  demonstrated: 

-  Improved  performance  of  the  SINCGARS  waveform  that 
met  requirements  of  mounted  and  dismounted  transmission 
range,  voice  quality,  and  call  completion  rates  under  benign 
conditions  of  developmental  test. 

-  Poor  reliability  with  the  SRW  waveform  demonstrating 
177  hours  MTBEFF  compared  to  the  Manpack  requirement 


of  477  hours.  This  translates  to  a  66  percent  chance  of 
completing  a  72-hour  mission  compared  to  a  requirement 
of  86  percent. 

•  The  Army  continues  preparation  for  a  future  Manpack  radio 
MOT&E-2,  which  will  include  competition  of  Program  of 
Record  and  alternate  vendors. 

•  The  JTRS  HMS  program  is  schedule-driven  and  has  reduced 
time  for  developmental  testing  to  support  an  aggressive 
operational  test  schedule. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  JTRS  HMS 
program  did  not  address  the  previous  recommendations  to 
perform  adequate  developmental  testing  prior  to  operational 
testing  and  to  complete  necessary  documentation  to  support 
developmental  and  operational  testing. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Ensure  that  adequate  developmental  testing  is  performed 
prior  to  future  operational  tests. 

2.  Correct  any  JTRS  HMS  deficiencies  noted  at  the  May  2012 
Manpack  radio  MOT&E  prior  to  the  scheduled  MOT&E-2. 

3.  Perfonn  a  holistic  reliability  growth  analysis  to  rigorously 
assess  Manpack  radio  maturity  and  to  provide  the 
information  needed  to  develop  a  detailed  plan  for  achieving 
required  reliability. 

4.  Complete  necessary  Manpack  radio  documentation  to 
support  future  developmental  and  operational  testing. 
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Joint  Tactical  Radio  System  (JTRS) 
Handheld,  Manpack,  and  Small  Form  Fit  (HMS) 

Rifleman  Radio 


Executive  Summary 

•  In  J anuary  2011,  the  Army  conducted  a  Verification  of 
Correction  of  Deficiencies  (VCD)  test  with  a  redesigned 
version  of  the  Rifleman  Radio.  During  the  VCD  test, 

Soldiers  demonstrated  that  the  redesigned  Rifleman  Radio 
corrects  most  of  the  radio’s  deficiencies  and  provides  some 
improvement  to  reliability. 

•  In  May  2011,  the  Joint  Tactical  Radio  System  (JTRS) 
Handheld,  Manpack,  and  Small  Form  Fit  (HMS)  program 
received  a  Milestone  C  Low-Rate  Initial  Production 
(LRIP)  decision  based  upon  the  improved  performance  the 
Rifleman  Radio  demonstrated  during  the  VCD.  The  Defense 
Acquisition  Executive  (DAE)  approved  the  Rifleman  Radio 
LRIP  quantity  of  6,250  radios. 

•  In  November  2011,  the  Army  conducted  a  Rifleman  Radio 
IOT&E  intended  to  support  a  Full-Rate  Production  decision 
as  a  part  of  the  Network  Integration  Evaluation  12.1.  DOT&E 
assessed  the  Rifleman  Radio  to  be  operationally  effective  with 
poor  reliability. 

•  From  February  through  April  2012,  the  Army  conducted 
Governmental  Developmental  Test  (GDT)  2.3  and  GDT  2.3a 
to  complete  developmental  testing  normally  completed  prior  to 
IOT&E.  During  GDT  2.3,  the  Rifleman  Radio  demonstrated 
new  failures  not  observed  in  IOT&E.  The  program  performed 
a  software  update  and  demonstrated  fixes  during  GDT  2.3a. 
The  Rifleman  Radio  showed  improvement  but  did  not  meet  its 
reliability  requirement. 

•  In  May  2012,  the  DAE  decided  to  pursue  a  competitive 
Full-Rate  Production  decision  and  approved  a  second  LRIP 
decision  for  13,077  radios. 

•  The  JTRS  HMS  program  is  schedule-driven  and  did  not 
complete  developmental  testing  prior  to  IOT&E  to  support  an 
aggressive  operational  test  schedule.  Operational  testing  has 
continued  to  reveal  problems  that  should  have  been  discovered 
and  fixed  during  developmental  testing. 

System 

•  JTRS  is  a  family  of  software-programmable  and  hardware 
configurable  digital  radios  intended  to  provide  increased 
interoperability,  flexibility,  and  adaptability  to  support 
numerous  tactical  communications  requirements. 

•  The  JTRS  HMS  program  provides  handheld  and  two-channel 
Manpack  radios  supporting  Army,  Marine  Corps,  Navy,  and 
Air  Force  operations.  The  program  develops  Small  Form  Fit 
(SFF)  radio  configurations  that  include  the  stand-alone  Army 
Rifleman  Radio  and  embedded  SFF  variants  that  serve  in 


Army  host  platforms  such  as  the  SFF-B  (Shadow  Unmanned 
Aerial  Vehicle),  SFF-B  (V)l  (Nett  Warrior),  and  the  SFF-D 
(Small  Unmanned  Ground  Vehicle). 

•  The  program  strategy  has  two  phases  of  HMS  production.  The 
JTRS  HMS  program  developed  the  Rifleman  Radio  as  part 

of  its  Phase  1  effort  to  provide  software  programmable  radios 
with  National  Security  Agency  (NSA)  Type  2  encryption 
of  unclassified  information  that  could  operate  a  networking 
waveform.  Phase  2  consists  of  developing  the  Manpack  radio 
to  provide  software  programmable  radios  with  NSA  Type  1 
encryption  of  classified  information. 

•  The  Rifleman  Radio  is  a  one-channel  radio  with  commercial 
GPS  that: 

-  Is  capable  of  operating  at  various  transmission  frequencies 
using  the  Soldier  Radio  Waveform  (SRW) 

-  Operates  at  5  watts  maximum  power  output 

-  Allows  Soldiers  to  participate  in  Army  doctrinal  voice 
networks  and  transmit  Position  Location  Information 

Mission 

Army  leaders  and  Soldiers  use  Rifleman  Radios  to  communicate 
and  create  networks  to  exchange  voice,  video,  and  data  using  the 
SRW  during  all  aspects  of  military  operations. 

Major  Contractor 

General  Dynamics,  C4  Systems  -  Scottsdale,  Arizona 
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Activity 

•  The  JTRS  HMS  program  completed  a  redesign  of  the 
Rifleman  Radio  hardware  and  improved  its  software  to 
address  deficiencies  identified  during  the  2009  Limited  User 
Test.  The  redesigned  Rifleman  Radio  features  improvements 
in  size,  weight,  battery  life,  and  increased  radio  frequency 
power  output. 

•  In  January  2011,  the  Army  conducted  a  Rifleman  Radio  VCD 
test  at  Fort  Benning,  Georgia,  to  demonstrate  improvements 
in  the  redesigned  radio. 

•  In  May  2011,  the  DAE  approved  a  Milestone  C  LR1P 
decision  to  procure  6,250  Rifleman  Radios  of  a  total 
acquisition  objective  of  193,279  radios. 

•  From  October  to  November  2011,  the  Army  Test  and 
Evaluation  Command  conducted  the  Rifleman  Radio  IOT &E 
at  White  Sands  Missile  Range,  New  Mexico,  as  part  of  the 
Army’s  Network  Integration  Evaluation  12.1.  The  test  was 
conducted  in  accordance  with  a  DOT&E -approved  test  plan. 
Operational  units  tested  the  Rifleman  Radio  using  the  Soldier 
Radio  Waveform  Network  Manager  (SRWNM)  to  plan  and 
load  SRW  network  configurations  into  the  radios.  Two 
platoons  of  infantry  engaged  in  a  variety  of  mission  scenarios 
employed  the  Rifleman  Radio.  An  additional  baseline 
infantry  platoon,  equipped  with  legacy  radios,  completed 
similar  missions  for  comparison  purposes. 

•  February  through  March  2012,  the  Army  conducted  the 
Rifleman  Radio  GDT  2.3  at  the  Electronic  Proving  Ground 
at  Fort  Huachuca,  Arizona.  The  Army  conducted  this  GDT 
to  complete  developmental  testing  that  the  Army  should  have 
completed  prior  to  IOT&E. 

•  In  April  2012,  the  Army  conducted  a  follow-on 
developmental  test,  GDT  2.3a.  The  Army  used  this  follow-on 
event  to  confirm  fixes  to  deficiencies  observed  during 

GDT  2.3. 

•  On  May  23,  20 12,  the  DAE  approved  a  second  LRIP  for  an 
additional  13,077  Rifleman  Radios  to  increase  the  total  LRIP 
procurement  to  10  percent  of  the  total  acquisition  objective. 

•  The  Army  continues  preparation  for  a  future  Rifleman  Radio 
IOT&E-2  that  will  include  competition  of  Program  of  Record 
and  alternate  radio  vendors. 

Assessment 

•  During  the  2009  Rifleman  Radio  Limited  User  Test,  DOT&E 
assessed  the  radio  as  useful  during  mission  preparation, 
movement,  and  reconnaissance  activities.  The  radio 
demonstrated  poor  performance  during  combat  operations 
and  poor  reliability. 

•  During  the  2011  Rifleman  Radio  VCD  test,  the  redesigned 
radio  demonstrated  the  following  improvements: 

-  Operational  reliability  of  277  hours  Mean  Time  Between 
Essential  Function  Failure  (MTBEFF)  compared  to  the 
revised  requirement  of  477  hours.  This  translates  to 
a  92  percent  chance  of  completing  a  24-hour  mission 
compared  to  a  requirement  of  95  percent. 


-  Transmission  range  met  the  radio’s  requirement  of 
2,000  meters  in  an  urban  setting  and  1,000  meters  in 
dense  vegetation. 

-  Radio  battery  life  exceeded  the  radio’s  revised  8-hour 
requirement. 

•  During  the  2011  Rifleman  Radio  IOT&E,  the 
production-representative  radio  demonstrated  the  following: 

-  Usefulness  in  supporting  combat  leaders  and  Soldiers  in  a 
wide  variety  of  missions. 

-  Enhanced  ability  for  Soldiers  to  execute  mission 
command  and  communicate  using  voice  and  data. 

-  Increased  effective  radio  transmission  range  relative  to 
legacy  radios. 

-  Reduced  reliability  of  1 6 1  hours  MTBEFF  compared 
to  the  requirement  of  477  hours.  This  translates  to  an 
86  percent  chance  of  completion  of  a  24-hour  mission 
compared  to  a  requirement  of  95  percent. 

-  Vulnerabilities  in  Information  Assurance  and  electronic 
warfare. 

•  During  the  2011  Rifleman  Radio  IOT&E,  the  SRWNM  did 
not  support  the  unit’s  mission  in  planning  and  loading  radios 
in  a  timely  manner  due  to  cumbersome  software  and  poor 
training  provided  to  Soldiers. 

•  During  the  2012  GDT  2.3,  the  Rifleman  Radio  demonstrated 
reliability  of  157  hours  MTBEFF.  This  reduced  reliability 
was  due  to  new  failures  (not  observed  in  IOT&E)  that 
included  spontaneous  reboots,  loss  of  the  ability  to  transmit, 
or  loss  of  the  ability  to  receive.  The  program  attributed 
these  failures  to  software  updates.  The  JTRS  HMS  program 
updated  the  software  and  performed  GDT  2.3a  to  demonstrate 
fixes. 

•  The  Rifleman  Radio  during  the  GDT  2.3  and  2.3a 
demonstrated  the  following  results: 

-  Estimated  reliability  of  the  Rifleman  Radio  is  3 10  hours 
MTBEFF,  which  is  a  93  percent  chance  of  completing  a 
24-hour  mission  against  a  95  percent  requirement. 

-  Message  completion  rate  of  84  percent  and  a  call 
completion  rate  of  95  percent.  The  Army’s  requirement 
for  both  is  90  percent. 

-  The  radio  met  its  requirements  for  Position  Location 
Information  updates,  size,  weight,  power,  and  retention  of 
cryptographic  information. 

•  The  Rifleman  Radio  program  is  schedule-driven.  As  a  result, 
the  Army  did  not  perform  the  necessary  developmental 
testing  required  to  ensure  success  during  the  Rifleman 
Radio  IOT&E.  Although  the  first  developmental  test  event 
was  conducted  prior  to  IOT&E,  the  Army  conducted  the 
remaining  two  developmental  test  events  several  months  after 
the  operational  test. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  JTRS  HMS 
program  did  not  address  the  previous  recommendations  to 
perform  adequate  developmental  testing  prior  to  operational 
testing  and  to  complete  necessary  documentation  to  support 
future  developmental  and  operational  testing. 

•  FY12  Recommendations.  The  Army  should  address  the 
previous  recommendations  and: 

1 .  Ensure  that  adequate  developmental  testing  is  performed 
prior  to  future  operational  tests. 

2.  Complete  necessary  Rifleman  Radio  documentation  to 
support  future  developmental  and  operational  testing. 
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Joint  Tactical  Radio  System  (JTRS) 

Network  Enterprise  Domain  (NED)  Network  Managers 


Executive  Summary 

•  Joint  Tactical  Radio  System  (JTRS)  Enterprise  Network 
Manger  (JENM)  runs  both  the  JTRS  Wideband  Networking 
Waveform  (WNW)  Network  Manager  (JWNM)  and 

the  Soldier  Radio  Waveform  (SRW)  Network  Manager 
(SRWNM).  JENM  allows  signal  personnel  to  manage  the 
networks  of  JTRS  software-defined  radio  sets. 

•  Of  the  JENM  management  functions  (planning,  loading, 
monitoring,  controlling,  and  reporting)  SRWNM  and 
JENM/SRWNM  testing  examined  the  planning,  loading,  and 
monitoring  functions. 

SRWNM 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  operational  testing  from 
October  through  November  2011.  Based  on  results  from 
testing,  DOT&E  issued  an  Operational  Assessment  report 
in  August  2012  assessing  SRWNM  as  not  capable  of 
supporting  the  unit’s  mission  and  unreliable  in  supporting 
JTRS  Rifleman  Radio.  The  SRWNM  lacks  the  capability 
needed  for  Soldiers  to  use  it  quickly  and  effectively  to 
configure  a  network.  The  Army  had  not  developed  the 
tactics,  techniques,  and  procedures  needed  for  Soldiers  to 
employ  the  SRWNM  in  support  of  units  using  Rifleman 
Radios  during  combat  operations. 

JENM/SRWNM 

•  In  a  July  2012  Operational  Assessment  report,  DOT&E 
assessed  JENM/SRWNM  as  effective  and  suitable  in 
support  of  JTRS  Manpack  radios  operating  the  SRW 

in  part  due  to  applying  the  lessons  learned  from  the 
SRWNM  IOT&E.  Soldiers  also  used  JENM/SRWNM  to 
successfully  load  Rifleman  Radios  as  part  of  the  Army’s 
Network  Integration  Evaluation  (NIE)  activities.  Lessons 
learned  from  the  SRWNM  IOT&E  applied  during  the 
JENM/SRWNM  IOT&E  included  improvements  to  Soldier 
training,  development  and  validation  of  the  network  plan, 
and  concept  of  operations  for  loading  and  configuring  the 
Manpack  radios  for  the  MOT&E. 

•  The  test  unit  successfully  used  JENM/SRWNM  to  import 
a  network  plan,  load  the  radios,  and  monitor  the  SRW 
network.  Flowever,  Soldiers  reported  the  monitoring 
capability  had  limited  utility. 

System 

•  JTRS  Network  Enterprise  Domain  (NED)  software 
applications  allow  the  JTRS  software-defined  radio  sets  to 
provide  communications  to  tactical  forces.  The  software 
applications  include  waveforms,  enterprise  networking 
services  (route  and  retransmission  among  waveforms),  and 
enterprise  network  management. 


The  waveforms  and  enterprise  networking  services  software 
are  integrated  into  and  are  considered  part  of  a  JTRS  radio 
set,  and  their  performance  is  part  of  that  reported  for  the 
JTRS  Handheld,  Manpack,  and  Small  Form  Fit  (HMS);  and 
Airborne,  Maritime,  and  Fixed  Station  (AMF)  radio  products. 
The  enterprise  network  management  software  is  separate  from 
the  JTRS  radio  sets  and  is  deployed  on  designated  commercial 
off-the-shelf  laptop  computers. 

-  The  current  network  manager  products  are:  JWNM  for 
managing  WNW  networks;  and  SRWNM  for  managing 
SRW  networks  of  JTRS  software-defined  radio  sets. 

-  In  FY12,  the  JTRS  NED  Program  Office  integrated 
SRWNM  onto  the  JENM.  In  the  near  future,  the  Program 
Office  intends  to  integrate  both  JWNM  and  SRWNM  onto 
a  single  JENM  laptop  computer. 

-  Enterprise  network  management  functions  include 
planning,  loading,  monitoring,  controlling,  and  reporting. 

-  The  planning  function  develops  the  network  parameters 
and  creates  a  Radio  Mission  Data  Set  file. 

-  The  loading  function  transfers  the  Radio  Mission  Data  Set 
file  into  the  HMS  or  AMF  radio  sets  to  configure  them. 

-  The  monitoring  function  provides  a  near  real-time  display 
of  the  WNW  or  SRW  network  status  and  the  conditions  of 
the  radios. 

-  The  controlling  function  allows  the  signal  Soldier  to  make 
changes  to  the  network,  to  include  sending  commands  to 
the  radio  operator,  changing  the  configuration  parameters 
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of  the  radio  sets,  or  conducting  cryptographic  functions 
(rekey,  zeroize,  and  transfer). 

-  The  reporting  function  records  all  network  management 
events  and  makes  the  data  available  for  analysis. 

Mission 

•  Forward-deployed  military  forces  use  JTRS  radios  to 
communicate  and  create  networks  to  exchange  voice,  video, 
and  data  during  all  aspects  of  tactical  military  operations. 

•  Signal  staffs  use  the  JENM  to  plan,  load,  monitor,  control,  and 
report  on  network  operations  involving  JTRS  HMS  and  AMF 


software-defined  radio  sets,  as  well  as  non-developmental  item 
radios,  running  WNW  and  SRW. 

Major  Contractors 

•  The  Boeing  Company,  Phantom  Works  Division  -  Huntington 
Beach,  California  (the  JWNM  and  JENM  developer) 

•  ITT  Electronics  Systems  Division  -  Clifton,  New  Jersey  (the 
SRWNM  developer) 


Activity 

•  DOT&E  approved  the  JTRS  NED  Test  and  Evaluation  Master 

Plan  in  July  2011. 

SRWNM 

•  DOT&E  approved  the  SRWNM  IOT&E  plan  on 
October  7,  2011. 

•  COTF  conducted  the  SRWNM  IOT&E  in  support  of 
the  Rifleman  Radio  IOT&E  from  October  through 
November  2011.  Problems  with  the  network  plan  and  use 
of  SRWNM  began  to  affect  the  start  date  for  the  Rifleman 
Radio  IOT&E.  Contractor  support  was  used  to  correct  the 
plan  and  reload  the  radios  to  ensure  the  Rifleman  Radio  test 
would  start  on  time.  As  a  result,  the  SRWNM  test  was  not 
conducted  in  accordance  with  the  DOT&E-approved  test 
plan. 

JENM/SRWNM 

•  DOT&E  approved  the  JENM/SRWNM  IOT&E  plan  on 
March  29,  2012. 

•  COTF  conducted  the  JENM/SRWNM  IOT&E  as  described 
in  the  DOT&E-approved  test  plan  in  support  of  the 
Army’s  NIE  12.2  and  Manpack  radio  IOT&E  in  April  and 
May  2012. 

•  The  JTRS  NED  Program  Office  integrated  the  SRWNM 
software  onto  the  JENM  platform  following  the  Rifleman 
Radio  IOT&E  (referred  to  as  JENM/SRWNM). 

JENM 

•  The  JTRS  NED  Program  Office  continues  development  of 
the  JENM,  which  will  integrate  the  JWNM  and  SRWNM 
into  a  single  network  management  product  in  FY13. 

Assessment 

SRWNM 

•  SRWNM  developmental  testing  in  FY 1 1  confirmed  the 
system’s  capability  to  plan  networks,  create  mission  data 
sets,  and  then  load  mission  data  sets  for  the  Rifleman 
Radios  loaded  with  the  SRW.  Per  the  Army  concept  of 
operations,  there  is  no  requirement  for  SRWNM  to  monitor 
the  Rifleman  Radio/ SRW  network,  so  this  function  was  not 
demonstrated  in  the  SRWNM  IOT&E. 

•  DOT&E  assessed  SRWNM  as  not  capable  of  supporting  the 
unit’s  mission  and  the  system  demonstrated  poor  reliability 


in  support  of  the  Rifleman  Radio.  Soldiers  could  not  use 
SRWNM  to  quickly  and  effectively  configure  a  network. 
The  Army  had  not  developed  adequate  training  or  the 
necessary  tactics,  techniques,  and  procedures  for  Soldiers 
to  employ  the  SRWNM  in  support  of  units  using  Rifleman 
Radios  during  combat  operations. 

•  Shortfalls  in  the  capability  and  use  of  SRWNM  include  the 
following: 

-  Lack  of  troubleshooting  procedures 

-  Lack  of  a  network  plan  validation  event  or  process 

-  Excessive  time  needed  to  load  Rifleman  Radios 

-  Lack  of  procedures  to  distribute  configured  radios  by 
mission 

-  Lack  of  flexible  radio  configurations  to  support  changing 
missions 

•  The  single  signal  Soldier  in  the  company  was  overwhelmed 
by  the  sheer  amount  of  communications  equipment  he  was 
responsible  for  operating  and  maintaining. 

JENM/SRWNM 

•  DOT&E  assessed  JENM/SRWNM  as  effective  and  suitable 
in  support  of  Manpack  radios  operating  the  SRW. 

•  Lessons  learned  from  the  SRWNM  IOT&E  were  applied 
to  the  training,  development,  and  validation  of  the  network 
plan  and  concept  of  operations  for  loading  and  configuring 
Manpack  radios.  Soldiers  using  JENM/SRWNM  also 
demonstrated  the  ability  to  successfully  load  Rifleman 
Radios  as  part  of  NIE  activities. 

-  The  test  unit  successfully  used  JENM/SRWNM  to 
import  a  network  plan,  load  the  radios,  and  monitor 
the  SRW  network.  However,  Soldiers  reported  the 
monitoring  capability  had  limited  utility. 

-  Operational  challenges  with  the  use  of  JENM/SRWNM 
still  exist.  The  process  for  planning  and  loading  a 
network  is  cumbersome  and  time  consuming  (about 
20-25  minutes  per  radio).  This  time  includes  several 
steps  that  include  radio  start  up,  loading  the  plan,  loading 
the  crypto  keys,  and  checking  the  communications  link. 
For  the  test,  the  company  had  46  Manpack  radios  and  96 
Rifleman  Radios.  In  total,  it  took  two  Soldiers  two  to 
three  days  to  load  and  check  all  the  radios. 
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•  Dynamic  task  organization  remains  an  operational 
deficiency  because  of  the  length  of  time  it  takes  to  change 
radio  presets  to  match  the  task  organization.  The  brigade 
estimated  it  would  take  48  hours  to  modify  the  radio  presets 
in  the  plan  and  reload  all  the  radios  with  the  new  plan  to 
allow  the  task  organization  of  a  single  platoon  to  a  different 
company. 

•  The  JENM/SRWNM  had  no  significant  reliability  problems. 
The  JENM/SRWNM  IOT&E  results  demonstrated  that 

the  software  and  hardware  improvements  corrected  the 
deficiencies  noted  in  the  previous  SRWNM  IOT&E. 
Training  was  much  better  than  that  experienced  in  the 
previous  SRWNM  IOT&E  based  on  Soldier  input,  but 
Soldiers  reported  that  they  would  like  to  see  more  practical 
exercises  and  documented  processes  in  future  training. 

•  The  single  signal  Soldier  in  the  company  was  overwhelmed 
by  the  sheer  amount  of  communications  equipment  he  was 
responsible  for  operating  and  maintaining. 

•  Development  and  verification  of  the  JTRS  networks 
is  a  significant  undertaking.  The  Army  must 
determine  who  is  going  to  develop  the  SRW  network 
architecture/communication  plans  and  translate  those 
into  JENM  files  and  radio  configuration  files  for  the 
multi-Service  units  to  be  equipped  with  Manpack  radios.  It 
is  not  clear  whether  the  Program  Office  can  sustain  the  level 


of  effort  required  to  construct  the  needed  inputs  for  each 
unit  as  JTRS  radios  are  fielded  across  the  force. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 
two  of  the  three  recommendations  for  FY11.  There  is  still 
no  movement  on  developing  an  integrated  test  methodology 
for  JENM  developmental  and  operational  testing.  With  the 
closing  of  the  JTRS  Joint  Program  Executive  Office  and  the 
radio  programs  now  all  under  separate  program  management, 
resolving  this  FY11  recommendation  is  critical  to  ensure 
JENM  is  properly  tested. 

•  FY 12  Recommendations.  The  Army  should: 

1 .  Evaluate  the  force  structure  implications  of  adding  JTRS 
radios  and  network  management  responsibilities  into 
company-level  organizations.  The  single  signal  Soldier 
in  the  company  was  overwhelmed  by  the  sheer  amount 
of  communications  equipment  he  was  responsible  for 
operating  and  maintaining. 

2.  Determine  who  is  going  to  develop  the  WNW  and  SRW 
network  architecture/communications  plans,  and  then 
translate  those  into  JENM  files  and  radio  configuration  files 
for  the  multi-Service  units  to  be  equipped  with  Manpack, 
Rifleman,  andAMF  radios. 
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Joint  Warning  and  Reporting  Network  (JWARN) 


Executive  Summary 

•  Joint  Warning  and  Reporting  Network  (JWARN)  software 
hosted  on  Global  Command  and  Control  System 
(GCCS)  -  Maritime  is  an  operationally  effective  automated 
Chemical,  Biological,  Radiological,  and  Nuclear  (CBRN) 
warning,  reporting,  and  analysis  tool  for  Navy  operations  to 
send  warning  reports  to  units  at  risk  from  a  CBRN  hazard  in 
time  to  take  protective  action  when  the  units  are  more  than 
10  kilometers  downwind.  The  Navy  should  upgrade  the 
GCCS  -  Joint  version  currently  employed  at  its  Maritime 
Operations  Centers  to  GCCS  -  Joint  with  Plain  Language 
Address  (PLA)  capability  to  utilize  JWARN  to  send  timely 
warning  messages  between  battle  groups  and  to  other  Services 
when  ships  are  employing  emissions  control  procedures. 

•  Based  on  FOT&E  results,  JWARN  software  hosted  on 
GCCS  -  Maritime  is  operationally  suitable  and  reliable. 
Operators  with  JWARN  new  equipment  training  are  able 
to  successfully  execute  basic-level  analysis  and  reporting 
scenarios.  During  FOT&E,  JWARN  operators  lost  situational 
awareness  during  more  complex  CBRN  attack  scenarios  and 
did  not  send  appropriate  and  timely  hazard  warning  reports. 

•  The  JWARN  Program  Office  shoidd  develop  and  field 
computer-based  training  that  includes  basic  to  advanced 
scenario  exercises  to  increase  operator  skills  and  provide 
sustainment  training. 

System 

•  JWARN  is  a  joint  automated  CBRN  warning,  reporting, 
and  analysis  software  tool  that  resides  on  joint  and  Service 
command  and  control  systems  such  as  the  GCCS  -  Army, 
GCCS  -  Joint,  GCCS  -  Maritime,  and  Command  and  Control 
Personal  Computer/Joint  Tactical  Common  Workstation,  or 
stand-alone  computers. 

•  JWARN  software  automates  the  NATO  CBRN  warning 
and  reporting  process  to  increase  the  speed  and  accuracy 


of  information  sharing  to  support  force  protection  decision 
making  and  situational  awareness. 

•  JWARN  uses  the  Common  Operating  Picture  of  the  host 
command  and  control  network  to  display  ground  maps; 
unit  locations;  location  of  CBRN  events;  and  the  predicted 
or  actual  location  of  hazards  to  support  the  Commanders’ 
situational  awareness  and  response  capability. 

Mission 

JWARN  operators  in  command  cells  support  CBRN  force 
protection,  battlefield  management,  and  operational  planning  by 
predicting  chemical,  biological,  and  nuclear  hazard  areas  based 
on  sensor  and  observer  reports,  identifying  affected  units  and 
operating  areas,  and  transmitting  warning  reports. 

Major  Contractor 

Northrop  Grumman  Mission  Systems  -  Orlando,  Florida 


Activity 

•  The  Army  Test  and  Evaluation  Command  and  the  Navy’s 
Commander,  Operational  Test  and  Evaluation  Force  conducted 
follow-on  operational  testing  of  the  JWARN  hosted  on 
GCCS  -  Maritime  from  June  25  -  29,  2012,  at  the  Space  and 
Naval  Warfare  Systems  Center  Pacific  computer  laboratory,  in 
accordance  with  the  DOT&E-approved  operational  test  plan. 

•  DOT&E  approved  an  updated  Test  and  Evaluation  Master 
Plan  on  March  19,  2012,  to  address  follow-on  developmental 
and  operational  testing  of  the  JWARN  when  hosted  on 
GCCS  -  Maritime. 


•  The  program  conducted  developmental  and  reliability  growth 
testing  to  demonstrate  readiness  for  follow-on  operational 
testing  at  the  Space  and  Naval  Warfare  Systems  Center  Pacific 
computer  laboratory  in  San  Diego,  California,  from  November 
to  December  2011  with  JWARN  hosted  on  GCCS  -  Maritime 
and  GCCS  -  Joint  with  PLA. 

•  DOT&E  approved  the  JWARN  follow-on  operational  test  plan 
on  June  5,  2012. 
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Assessment 

•  JWARN  software  hosted  on  GCCS  -  Maritime  provides  an 
operationally  effective  automated  CBRN  warning,  reporting, 
and  analysis  tool  to  send  warning  reports  to  units  at  risk  from  a 
CBRN  hazard  in  time  to  take  protective  action  if  the  units  are 
more  than  10  kilometers  downwind  for  the  initial  hazard. 

•  During  normal  operations,  U.S.  Navy  Expeditionary  Strike 
Groups  can  exchange  CBRN  hazard  warnings  and  reports 
with  Navy  Maritime  Operations  Centers  for  communication 
across  the  naval  fleet  and  Services  using  JWARN  and  GCCS 
host  e-mail  capability.  When  ships  are  employing  emissions 
control  procedures  to  avoid  being  located  by  the  enemy,  e-mail 
capability  is  shut  down.  Ships  in  emissions  control  mode  are 
able  to  send  CBRN  JWARN  hazard  warnings  and  reports  using 
GCCS-Maritime  PLA  text  messaging  capability. 

•  The  version  of  GCCS  -  Joint  used  by  Navy  Maritime 
Operations  Centers  does  not  have  PLA  capability.  JWARN 
is  not  compatible  with  the  current  legacy  software  used  by 
Navy  Maritime  Operations  Centers  to  send  PLA  text  messages. 
Personnel  at  the  Navy’s  Maritime  Operations  Centers  must 
manually  enter  information  from  JWARN  reports  into  a  legacy 
system  with  PLA  capability  to  send  CBRN  warnings  and 
reports  to  ships  employing  emission  control  procedures.  This 
presents  an  opportunity  for  transcription  errors  and  creates  a 
delay  that  may  cause  CBRN  warning  messages  to  reach  ships 
at  risk  too  late  to  implement  effective  protective  actions. 

•  During  FOT&E,  operators  using  JWARN  demonstrated 
interoperability  with  a  developmental  version  of  GCCS  -  Joint 


with  PLA  capability.  Fielding  GCCS  -  Joint  with  PLA 
capability  at  Navy  Maritime  Operations  Centers  would 
speed  delivery  of  CBRN  warning  messages  when  ships  are 
implementing  emissions  control  measures  and  are  unable  to 
receive  e-mail  communications. 

•  JWARN  software  hosted  on  GCCS  -  Maritime  is  operationally 
suitable.  The  JWARN  software  hosted  on  GCCS  -  Maritime 
is  reliable.  Operators  with  new  equipment  training  are  able 

to  successfully  execute  basic-level  analysis  and  reporting 
scenarios.  However,  JWARN  operators  require  advanced 
training  in  order  to  successfully  execute  CBRN  analysis  and 
reporting  when  faced  with  complex  CBRN  attack  scenarios. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  manager 
addressed  all  previous  recommendations. 

•  FY12  Recommendations. 

1.  The  Navy  should  work  with  the  GCCS  program  to  coordinate 
deployment  of  GCCS  -  Joint  with  PLA  capability  to  Maritime 
Operations  Centers  with  deployment  of  GCCS  -  Maritime 
(with  JWARN). 

2.  The  program  manager  should  develop  and  field 
computer-based  training  that  includes  basic  to  advanced 
scenario  exercises  to  increase  operator  skills  and  provide 
sustainment  training. 
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Key  Management  Infrastructure  (KMI) 


Executive  Summary 

•  In  January  2012,  the  Key  Management  Infrastructure 
(KMI)  Program  Management  Office  (PMO)  and  DoD  Chief 
Information  Officer  (CIO)  declared  an  acquisition  cost  and 
schedule  threshold  breach,  resulting  in  a  Critical  Change 
Review  of  the  program.  Subsequently,  the  Chairman  of 
the  Joint  Chiefs  of  Staff  declared  KMI  essential  to  national 
security  and  OSD  recertified  the  program  to  Congress. 

•  The  Joint  Interoperability  Test  Command  (JITC)  conducted 
an  IOT&E  from  July  until  August  2012.  The  results  were  a 
marked  improvement  over  previous  operational  assessments; 
however,  there  were  still  several  operational  effectiveness  and 
suitability  problems  uncovered  during  the  testing  event  that 
must  be  corrected  before  continued  deployment. 

•  The  KMI  transition  process  and  Services’  fielding  concepts 
must  be  matured  to  ensure  an  accurate  and  smooth  migration 
from  the  legacy  Electronic  Key  Management  System  (EKMS) 
to  KMI.  Configuration  management  controls  and  training 

of  personnel  at  the  KMI  operational  support  site  need 
improvement  to  eliminate  system  inconsistencies. 

•  The  new  Type  1  token  hardware  and  its  stability  are  an 
improvement  over  previous  tokens,  but  its  reliability  is 
deficient  and  further  product  refinement  and  testing  is 
necessary  for  a  suitability  determination. 

•  KMI  is  potentially  operationally  effective  once  significant 
transition  problems  are  resolved.  Security  is  undetermined, 
pending  a  Red  Team’s  assessment.  KMI  is  interoperable 
and  received  full  certification  for  Spiral  1 ;  however,  KMI 
was  determined  to  be  unsuitable  due  to  the  lack  of  help 
desk  preparedness  for  operational  support,  deficient  token 
reliability,  and  immature  Configuration  Control  Board  and 
configuration  management  processes. 

•  Despite  problems  identified  during  operational  testing,  the 
KMI  program  continues  to  show  progress  toward  delivering 
a  useful  cryptographic  capability  for  system  managers  and 
users.  Operational  users  in  the  IOT&E  reviewed  the  system 
capabilities  positively,  once  the  transition  process  completed. 

System 

•  KMI  is  intended  to  replace  the  legacy  EKMS  to  provide 
a  means  for  securely  ordering,  generating,  producing, 
distributing,  managing,  and  auditing  cryptographic  products 
(e.g.,  asymmetric  key,  symmetric  keys,  manual  cryptographic 
systems,  and  cryptographic  applications). 

•  KMI  Spiral  1  consists  of  core  nodes  that  provide  web 
operations  at  a  single  site  operated  by  the  National  Security 
Agency  (NSA),  as  well  as  individual  client  nodes  distributed 
globally  to  provide  secure  key  and  software  provisioning 
services  for  the  DoD,  intelligence  community,  and  agencies. 
Spiral  2  will  provide  improved  capability  through  software 
enhancements  to  the  Spiral  1  baseline. 
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•  KMI  combines  substantial  custom  software  and  hardware 
development  with  commercial  off-the-shelf  (COTS)  computer 
components.  The  custom  hardware  includes  an  Advanced  Key 
Processor  for  autonomous  cryptographic  key  generation  and 

a  Type  1  user  token  for  role-based  user  authentication.  The 
COTS  components  providing  user  operations  include  a  client 
host  computer.  High  Assurance  Internet  Protocol  Encryptor 
(KG-250),  monitor,  keyboard,  mouse,  printer,  and  barcode 
scanner. 

Mission 

•  Combatant  Commands,  Services,  DoD  agencies,  other  Federal 
government  agencies,  coalition  partners,  and  allies  will  use 
KMI  to  provide  secure  and  interoperable  cryptographic  key 
generation,  distribution,  and  management  capabilities  to 
support  mission-critical  systems,  the  Global  Information  Grid, 
and  initiatives  such  as  Cryptographic  Modernization. 

•  Service  members  will  use  KMI  cryptographic  products  and 
services  to  enable  security  services  (confidentiality,  non 
repudiation,  authentication,  and  source  authentication)  for 
diverse  systems  such  as  Identification  Friend-or-Foe,  GPS, 
Advanced  Extremely  High  Frequency  Satellite  System, 

Joint  Tactical  Radio  System,  and  Warfighter  Information 
Network  -  Tactical. 

Major  Contractors 

•  SAIC  -  Columbia,  Maryland  (Spiral  2  Prime) 

•  General  Dynamics  Information  Assurance  Division  -  Needham, 
Massachusetts  (Spiral  1  Prime) 

•  BAE  Systems  -  Linthicum,  Maryland 

•  L3  Systems  -  Camden,  New  Jersey 

•  SafeNet  -  Belcamp,  Maryland 

•  Praxis  Engineering  -  Annapolis  Junction,  Maryland 
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Activity 

•  DOT&E  published  the  KMI  Operational  Assessment-2  report 
in  mid-October  2011.  Based  on  that  report,  the  DoD  CIO 
approved  Milestone  C  and  authorized  the  KMI  program  to 
enter  the  Production,  Deployment,  and  Sustainment  phase  for 
Capability  Increment  2  in  late  October  2011. 

•  In  January  2012,  the  KMI  PMO  and  DoD  CIO  declared  an 
acquisition  cost  and  schedule  threshold  breach,  resulting  in 

a  Critical  Change  Review  of  the  program.  Subsequently,  the 
Chairman  of  the  Joint  Chiefs  of  Staff  declared  KMI  essential 
to  national  security  and  OSD  recertified  the  program  to 
Congress. 

•  With  the  DoD  CIO  certification  of  the  program,  the  KMI 
Program  Office  moved  forward  with  Spiral  1  IOT&E 
preparations  and  Spiral  2  Contract  Award. 

•  In  accordance  with  the  Acquisition  Decision  Memorandum, 
the  KMI  program  manager  implemented  a  token  reliability 
growth  program  and  conducted  accelerated  life  testing  of 
120  tokens  in  order  to  increase  confidence  in  token  reliability. 
The  tokens  achieved  86,000  combined  hours  of  testing  while 
undergoing  temperature  and  vibration  cycles  to  demonstrate 

a  10,000-hour  Mean  Time  Between  Failure  with  80  percent 
confidence. 

•  The  KMI  program  and  JITC  conducted  an  IOT&E 
in  accordance  with  a  DOT&E-approved  plan  from 

July  9  through  August  10,  2012,  at  17  separate  Service  and 
Agency  locations  across  the  United  States. 

•  DOT&E  published  the  KMI  IOT&E  report  in 
mid-October  2012. 

Assessment 

•  KMI  is  potentially  operationally  effective  once  significant 
transition  problems  are  resolved.  Security  is  undetermined, 
pending  a  Red  Team’s  assessment.  KMI  is  interoperable 
and  received  full  certification  for  Spiral  1 ;  however,  KMI 
was  determined  to  be  unsuitable  due  to  the  lack  of  help 
desk  preparedness  for  operational  support,  deficient  token 
reliability,  and  immature  Configuration  Control  Board  and 
configuration  management  processes. 

•  Successful  completion  of  IOT&E  was  required  for  a  Spiral  1 
client  node  fielding  decision. 

-  The  program’s  major  hardware  developmental  item,  the 
Advanced  Key  Processor,  is  performing  well  and  exceeds 
its  expected  reliability. 

-  The  Service  and  agency  users  perceived  KMI  as  a  major 
qualitative  improvement  over  the  legacy  EKMS. 

•  KMI  is  significantly  more  stable  and  usable  than  in  previous 
test  events,  although  there  are  still  effectiveness  and  suitability 
problems  with  transition  and  backend  support. 

•  Based  on  the  IOT&E,  problems  with  system  performance, 
data  errors,  manual  KG-250  and  virtual  private  network  tunnel 
configurations,  and  network  connectivity  associated  with 
account  transition  affected  the  efficacy  and  speed  of  migrating 
cryptographic  products  from  EKMS  to  KMI.  This  caused 


significant  delays  in  account  migration  across  Services  and 
agencies. 

•  The  5  percent  token  failure  rate  observed  during  the  test 
period  is  not  acceptable.  While  observations  indicate  that 
token  reliability  is  improved,  additional  token  reliability 
testing  to  gain  more  usage  hours,  square  wave  (power 
cycling),  and  mechanical  insertion  testing  is  required. 

•  Symmetric  cryptographic  key  ordering  is  working  well  in 
KMI;  however,  asymmetric  keys  are  ordered  within  KMI 
but  their  production  is  accomplished  external  to  the  system. 
While  this  is  an  early  delivery  solution  to  support  Service 
needs,  operational  users  can  still  effectively  perform  their 
duties. 

•  Suitability  concerns  persist  with  the  system  and  logistical 
support  documentation.  These  products  are  improved, 
but  they  require  refinement  to  support  a  fully  operational 
capability. 

•  KMI  help  desk  and  system  administration  personnel  were 
not  adequately  prepared  to  support  a  fully  operational  KMI 
system. 

-  The  monitoring  capabilities  and  knowledge  base  are 
immature  and  not  well-exercised.  Critical  support 
personnel  were  not  prepared  to  support  the  user 
community  during  transition  and  day-to-day  operations. 

-  The  backend  support  is  not  prepared  for  a  KMI  full 
fielding  effort.  Significant  effort  must  be  made  to  refine 
the  technical  support  processes  before  the  system  is  fully 
deployed  to  hundreds  of  user  accounts  and  client  systems. 

•  The  Configuration  Control  Board  processes  and  procedures 
for  updating  and  implementing  system  builds  and  maintaining 
a  consistent  software  baseline  are  not  refined  or  effectively 
implemented  to  support  the  Services  and  agencies. 

•  The  KMI  transition  process  and  Services’  fielding  concepts 
must  be  matured  to  ensure  an  accurate  and  smooth  migration 
from  the  legacy  EKMS  to  KMI.  Configuration  management 
controls  and  training  of  personnel  at  the  KMI  operational 
support  site  needs  improvement  to  eliminate  system 
inconsistencies. 

•  KMI  system  documentation,  procedures,  and  training  for 
technical  staff  and  help  desk  personnel  are  inadequate. 
However,  KMI  operational  users  indicated  the  training  was 
thorough  but  too  compressed.  The  Services  have  limited 
training  blocks  to  two  weeks  because  of  Reservist  and 
National  Guard  requirements. 

•  Despite  some  problems  identified  during  operational  testing, 
the  KMI  program  continues  to  show  steady  progress  toward 
delivering  a  useful  cryptographic  capability  for  system 
managers  and  users.  Operational  users  in  the  IOT&E 
reviewed  the  system  capabilities  positively,  once  the 
transition  process  completed. 

•  Based  on  the  IOT&E  results,  the  KMI  PMO  scheduled  an 
F OT &E  for  J anuary  2013. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  KMI 
PMO  satisfactorily  addressed  seven  of  nine  previous 
recommendations.  Additional  PMO  effort  is  required  to 
adequately  address  regression  testing  for  system  builds  and 
their  deployment  and  more  time  is  necessary  to  provide 
adequate  KMI  training. 

•  FY12  Recommendations.  The  KMI  PMO  should: 

1.  Require  the  developmental  contractors  to  routinely 
demonstrate  system  readiness  through  regression  testing 
before  releasing  software  upgrades  and  system  builds,  and 
then  only  as  approved  by  the  Configuration  Control  Board 
for  distribution  to  Services  and  agencies. 

2.  Review  Service  and  agency  deployment  methods  and  work 
jointly  to  automate  transition  functions,  such  as  KG-250 


and  virtual  private  network  tunnel  configurations,  to  reduce 
problems  and  minimize  network  setup  changes  and  remote 
troubleshooting  from  the  KMI  support  site. 

3.  Capture  and  refine  documentation  of  all  KMI  process 
adjustments  for  incorporation  in  system  and  user-level 
operating  guides. 

4.  Assure  that  KMI  training  includes  sufficient  hands-on 
equipment  time  to  allow  users  to  gain  more  system 
familiarity,  knowledge,  and  proficiency.  Additional  user 
and  manager-level  training  is  needed  to  ensure  that  users 
can  understand  the  KMI  processes  and  operate  the  system. 

5.  Conduct  additional  token  reliability  testing  incorporating 
more  usage  hours,  square  wave  (power  cycling),  and 
mechanical  insertion  tests. 


KMI  65 


DOD  PROGRAMS 


66 


DOD  PROGRAMS 


Mine  Resistant  Ambush  Protected  (MRAP) 

Family  of  Vehicles 


Executive  Summary 

•  The  program  procured  250  Navistar  Dash  Ambulances  to 
fulfill  an  urgent  need  to  provide  protected  transport  and  urgent 
medical  treatment  for  Army  units  in  Afghanistan. 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  completed 
the  Limited  User  Test  (LUT)  of  the  Dash  Ambulance  with 
the  Independent  Suspension  System  (ISS)  in  November  2011 
at  Yuma  Proving  Ground,  Arizona,  in  accordance  with  the 
DOT&E-approved  test  plan. 

•  DOT&E  provided  an  Operational  Assessment  of  the  Dash 
Ambulance  in  August  2012. 

•  The  Dash  Ambulance  is  not  operationally  effective  and  not 
operationally  suitable. 

-  The  patient  compartment  of  the  vehicle  is  small  and  the 
litter  births  are  not  long  enough  to  safely  accommodate 
litter  patients  taller  than  5  feet  1 1  inches.  A  unit  equipped 
with  the  Dash  Ambulance  cannot  provide  safe  emergency 
medical  care  and  transport  for  tall  casualties  in  close 
proximity  to  enemy  forces. 

-  The  small  interior  of  the  Dash  Ambulance  does  not  provide 
sufficient  space  for  medical  equipment  and  inhibits  the 
ability  of  the  medic  to  maneuver  within  the  compartment 
to  properly  treat  patients. 

-  Loading  patients  in  the  Dash  Ambulance  is  hampered  due 
to  difficulty  aligning  and  securing  the  litter  onto  the  litter 
rail  system. 

•  The  Dash  Ambulance  is  reliable.  During  the  Dash  Ambulance 
LUT,  the  vehicle  demonstrated  796  Mean  Miles  Between 
Operational  Mission  Failure  (MMBOMF)  versus  its 
operational  requirement  of  600  MMBOMF.  The  vehicle  can 
be  maintained  by  Soldiers  and  is  recoverable. 

•  The  Dash  Ambulance  is  survivable.  The  vehicle  met 
the  MRAP  Capabilities  Document  version  1 . 1  threshold 
underbody  and  under-wheel  blast  requirements. 

•  The  MRAP  program  procured  2,071  Caiman  Multi-Terrain 
Vehicle  (CMTV)  rolling  chassis  with  ISS  and  Caiman 
Underbody  Blasts  Kits  to  address  crew  vulnerability 

and  mobility  deficiencies.  This  variant  is  undergoing 
developmental  testing.  Underbody  blast  testing  of  an  early 
version  of  the  CMTV  showed  a  significant  reduction  of  the 
crew  to  injuries  over  the  baseline  Caiman. 

•  The  Joint  Program  Office  MRAP  will  transfer  management 
responsibilities  to  the  Services  on  October  2013. 

System 

•  MRAP  is  a  family  of  vehicles  designed  to  provide  increased 
crew  protection  and  vehicle  survivability  against  current 
battlefield  threats,  such  as  IEDs,  mines,  and  small  arms.  The 
DoD  initiated  the  MRAP  program  in  response  to  an  urgent 


NAVISTAR  Dash  Ambulance 


BAE  Caiman  Multi-Terrain  Vehicle  Category  II 


operational  need  to  meet  multi-Service  ground  vehicle 
requirements.  MRAP  vehicles  provide  improved  vehicle 
and  crew  survivability  over  the  High  Mobility  Multi-purpose 
Wheeled  Vehicle  (HMMWV).  The  MRAPs  are  employed  by 
units  in  current  combat  operations  in  the  execution  of  missions 
previously  accomplished  with  the  HMMWV.  This  report 
covers  two  MRAP  vehicles: 

-  Navistar  Dash  Ambulance  with  ISS 

-  BAE  CMTV  Category  II 

•  The  Navistar  Dash  ISS  Ambulance  variant  is  designed 
to  transport  up  to  two  litter  patients  or  four  ambulatory 
casualties. 

•  The  BAE  CMTV  Category  II  is  designed  to  transport 
10  persons  plus  1  gunner. 

•  MRAP  vehicles  incorporate  current  Service  command  and 
control  systems  and  counter-lED  systems.  MRAP  vehicles 
have  gun  mounts  with  gunner  protection  kits  capable  of 
mounting  any  one  of  a  variety  of  weapons  systems  such  as 
the  M240B  medium  machine  gun,  the  M2  .50  caliber  heavy 
machine  gun,  and  the  Mk  1 9  grenade  launcher. 
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Mission  Major  Contractors 

Units  equipped  with  CMTV  conduct  small  unit  combat  •  BAE  Tactical  Vehicle  Systems  (TVS)  -  Sealy,  Texas 

operations  such  as  mounted  patrol,  convoy  security,  troop,  and  •  Navistar  Defense  -  Warrenville,  Illinois 

cargo  transportation.  The  unit  equipped  with  the  MRAP  Dash 

Ambulance  variant  supports  the  conduct  of  medical  treatment 

and  evacuation. 


Activity 

•  The  MRAP  program  continued  to  acquire  and  test  enhanced 
capabilities  to  integrate  across  the  MRAP  family  of  vehicles. 

In  FY12,  the  major  capability  insertions  include  the  ambulance 
kits  for  the  Navistar  Dash  and  the  ISS  for  the  BAE  CMTV. 

•  The  MRAP  program  procured  2,071  CMTV  rolling  chassis 
with  ISS  and  Caiman  Underbody  Blasts  Kits  to  address 
crew  vulnerability  and  mobility  deficiencies.  This  variant  is 
undergoing  developmental  testing.  Live  Fire  testing  of  the 
CMTV  will  commence  in  2QFY13. 

•  The  program  procured  250  Navistar  Dash  Ambulances. 

•  In  November  2011,  ATEC  completed  a  LUT  of  the  Dash 
Ambulance  with  ISS  at  Yuma  Proving  Ground,  Arizona,  in 
accordance  with  the  DOT&E-approved  test  plan. 

•  DOT&E  provided  an  Operational  Assessment  of  the  Dash 
Ambulance  in  August  2012. 

•  The  program  is  developing,  procuring,  and  integrating  the 
Army  network  capabilities  onto  MRAP  vehicles. 

•  The  Joint  Program  Office  MRAP  will  transfer  management 
responsibilities  to  the  Services  in  October  2013. 

Assessment 

•  The  Dash  Ambulance  is  not  operationally  effective  and  not 
operationally  suitable.  The  patient  compartment  of  the  vehicle 
is  small  and  the  litter  births  are  not  long  enough  to  safely 
accommodate  litter  patients  taller  than  5  feet  1 1  inches.  A 
unit  equipped  with  the  Dash  Ambulance  cannot  provide  safe 
emergency  medical  care  and  transport  for  tall  casualties  in 
close  proximity  to  enemy  forces.  This  problem  should  have 
been  corrected  prior  to  the  LUT. 

•  The  small  interior  of  the  Dash  Ambulance  does  not  provide 
sufficient  space  for  medical  equipment  and  inhibits  the  ability 
of  the  medic  to  maneuver  within  the  compartment  to  properly 
treat  patients. 

•  Loading  patients  in  the  Dash  Ambulance  is  hampered  due  to 
difficulty  aligning  and  securing  the  litter  onto  the  litter  rail 

system. 

•  The  Dash  Ambulance  is  reliable.  During  the  Dash  Ambulance 
LUT,  the  vehicle  demonstrated  796  MMBOMF  versus  its 


operational  requirement  of  600  MMBOMF.  The  vehicle  can 
be  maintained  by  Soldiers  and  is  recoverable 

•  The  Dash  Ambulance  is  survivable.  The  vehicle  met 
the  MRAP  Capabilities  Document  version  1 . 1  threshold 
underbody  and  under-wheel  blast  requirements. 

•  Based  on  performance  during  developmental  testing,  the 
CMTV  cannot  stop  following  sustained  operations  in  muddy 
terrain.  The  program  suspended  developmental  testing  until 
the  program  identifies  and  implements  a  materiel  solution  to 
fix  the  brake  system. 

•  Endurance  testing  of  the  CMTV  is  ongoing  at  Yuma  Proving 
Ground,  Arizona,  in  all  conditions  except  wet  off-road. 
Underbody  blast  testing  of  an  early  version  of  the  CMTV 
showed  a  significant  reduction  of  the  crew  to  injuries  over 
the  baseline  Caiman.  The  CMTV  experienced  problems 
associated  with  air  conditioner,  tire,  and  new  cab  mount 
cracking  failures.  The  program  should  resolve  these  problems 
prior  to  conducting  FOT&E. 

•  Planning  for  Live  Fire  testing  of  the  CMTV  is  ongoing  and 
testing  will  commence  in  FY13. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  has  not 
addressed  the  recommendation  to  improve  the  cross-country 
mobility  and  system  reliability  of  the  Navistar  MRAP 
Recovery  Vehicle. 

•  F Y 12  Recommendations  for  the  Dash  Ambulance  and  the 
CMTV.  Prior  to  conducting  FOT&E,  the  program  should: 

1 .  Redesign  the  Dash  Ambulance  to  accommodate  litter 
patients  taller  than  5  feet  11  inches.  Review  the  installed 
medical  equipment  with  the  objective  of  providing 
additional  internal  space  and  reducing  patient  loading  time. 

2.  Fix  the  CMTV  brake  system  to  allow  the  vehicle  to  stop  in 
muddy  conditions  and  improve  the  overall  reliability  of  the 
CMTV. 
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Mine  Resistant  Ambush  Protected  (MRAP) 
All-Terrain  Vehicle  (M-ATV) 


Executive  Summary 

•  The  Mine  Resistant  Ambush  Protected  (MRAP)  All-Terrain 
Vehicle  (M-ATV)  with  Underbody  Improvement  Kit  (UIK) 
participated  in  the  Dash  MRAP  Ambulance  Limited  User  Test 
(LUT)  in  November  2011  at  Yuma  Proving  Ground,  Arizona, 
in  accordance  with  the  DOT&E-approved  test  plan. 

•  DOT&E  provided  an  Operational  Assessment  of  the  M-ATV 
with  UIK  in  August  2012. 

•  The  M-ATV  with  UIK  is  operationally  effective,  operationally 
suitable,  and  survivable. 

-  A  unit  equipped  with  the  M-ATV  with  UIK  can  accomplish 
tactical  transport  missions. 

-  The  M-ATV  with  UIK  provides  sufficient  armored 
mobility  to  conduct  missions  over  the  type  of  terrain 
typically  found  in  Afghanistan. 

-  The  M-ATV  with  UIK  demonstrated  off-road  mobility  and 
maneuver  capability  similar  to  the  base  M-ATV  during  the 
MRAP  Dash  Ambulance  LUT.  The  M-ATV  with  UIK  met 
its  reliability,  operational  availability,  and  maintainability 
requirements  based  on  the  LUT. 

-  The  M-ATV  with  UIK  meets  the  level  for  improved 
protection  against  underbody  blast  threats  specified  in  the 
Joint  Urgent  Operational  Need  Statement. 

•  In  October  2014,  the  Joint  Program  Office  (JPO)  MRAP  will 
transition  management  of  the  of  the  Special  Operations  Forces 
(SOF)  M-ATV  fleet  to  the  Services. 

•  United  States  Special  Operations  Command  (USSOCOM)  is 
planning  an  FOT&E  of  the  SOF  M-ATV  in  2QFY13. 

System 

•  The  M-ATV  is  designed  for  five  passenger  positions  including 
a  gunner.  The  vehicle  incorporates  current  Service  command 
and  control  and  counter-IED  systems.  The  M-ATV  includes 
gun  mounts  with  gunner  protection  kits  capable  of  mounting 

a  variety  of  weapons  systems  such  as  the  M240B  medium 
machine  gun,  the  M2  .50  caliber  heavy  machine  gun,  and  the 
Mk  19  grenade  launcher. 


M-ATV  with  Underbody  Special  Operations 

Improvement  Kit  (UIK)  Forces  M-ATV  UIK 


•  The  M-ATV  has  the  capability  to  add  protection 
against  attacks  by  explosively  formed  penetrators,  and 
rocket-propelled  grenades  to  support  mounted  patrols, 
reconnaissance,  security,  and  convoy  protection. 

•  USSOCOM  required  modifications  to  the  Army  M-ATV 
to  support  SOF  missions.  The  modifications  included  five 
passenger  positions  including  a  gunner,  protection  for  the 
cargo  area,  rear  area  access,  and  some  other  improvements  for 
human  factors. 

•  The  M-ATV  with  UIK  and  SOF  M-ATV  UIK  are  designed  to 
provide  improved  underbody  blast  protection  to  the  M-ATV 
variants. 

Mission 

Multi-service  and  special  operations  units  equipped  with  the 
M-ATV  conduct  mounted  patrols,  convoy  patrols,  convoy 
protection,  reconnaissance,  and  communications,  as  well  as 
command  and  control  missions  to  support  combat  and  stability 
operations  in  highly  restricted  rural,  mountainous,  and  urban 
terrain. 

Major  Contractor 

Oshkosh  Defense  -  Oshkosh,  Wisconsin 


Activity 

•  The  M-ATV  with  UIK  participated  in  the  Dash  MRAP 
Ambulance  LUT  in  November  2011  at  Yuma  Proving  Ground, 
Arizona,  in  accordance  with  the  DOT&E-approved  test  plan. 

•  In  FY12,  the  program  integrated  the  SOF-specific  UIK  onto 
the  SOF  M-ATV  fleet  to  improve  SOF  M-ATV  underbody 
blast  protection. 


•  DOT&E  provided  an  Operational  Assessment  of  the  M-ATV 
with  UIK  in  August  2012. 

•  USSOCOM  completed  the  developmental  test  weapons 
firing  event  of  the  SOF  M-ATV  with  the  Common  Remotely 
Operated  Weapon  Station  (CROWS)  at  Fort  Campbell, 
Kentucky,  in  April  2012  to  examine  the  capability  to  fire  the 


M-ATV  69 


DOD  PROGRAMS 


Mk  19  grenade  machine  gun  with  the  CROWS  integrated  on 
the  SOF  M-ATV  in  an  operational  environment. 

•  The  program  is  developing,  procuring,  and  integrating  the 
Army  network  capabilities  onto  base  M-ATVs. 

•  The  SOF  M-ATV  UIK  Live  Fire  testing  was  completed  in 
August  2012. 

•  In  October  2014,  the  JPO  MRAP  will  transition  management 
of  the  SOF  M-ATV  fleet  to  the  Services. 

•  USSOCOM  is  planning  an  FOT&E  of  the  SOF  M-ATV  in 
2QFY13. 

Assessment 

•  The  M-ATV  with  UIK  is  operationally  effective,  operationally 
suitable,  and  survivable.  A  unit  equipped  with  the  M-ATV 
with  UIK  can  accomplish  tactical  transport  missions.  The 
M-ATV  with  UIK  provides  sufficient  armored  mobility  to 
conduct  missions  over  the  type  of  terrain  typically  found  in 
Afghanistan. 

•  The  M-ATV  with  UIK  demonstrated  off-road  mobility  and 
maneuver  capability  similar  to  the  base  M-ATV  during  the 
MRAP  Dash  Ambulance  LUT.  The  M-ATV  with  UIK  met 


its  reliability,  operational  availability,  and  maintainability 
requirements  based  on  the  LUT. 

•  The  M-ATV  with  UIK  meets  the  level  for  improved  protection 
against  underbody  blast  threats  specified  in  the  Joint  Urgent 
Operational  Need  Statement. 

•  During  developmental  test  weapons  firing  of  the  SOF  M-ATV, 
no  CROWS/SOF  M-ATV  integration  failures  were  observed 
after  firing  approximately  480  rounds  of  high-explosive 
dual-purpose  ammunition.  Four  weapon-firing  failures  were 
attributed  to  operator  error  indicating  more  CROWS  and 
weapon  proficiency  training  is  needed  before  FOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  USSOCOM 
has  addressed  two  of  the  three  previous  SOF  M-ATV 
recommendations.  USSOCOM  did  not  address  the 
recommendation  related  to  improving  the  visibility  of  the  SOF 
passenger  by  installing  larger  rear  windows  in  SOF  M-ATV. 

•  FY12  Recommendation. 

1.  USSOCOM  should  continue  to  plan  and  conduct  the  SOF 
M-ATV  FOT&E. 
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Multi-functional  Information  Distribution  System  (MIDS) 
Joint  Tactical  Radio  System  (JTRS) 


Executive  Summary 

•  The  Navy  completed  the  Verification  of  Correction  of 
Deficiencies  (VCD)  operational  test  of  the  Multi-functional 
Information  Distribution  System  Joint  Tactical  Radio  System 
(MIDS  JTRS)  core  terminal  as  integrated  into  the  F/A-18E/F 
in  December  2011. 

•  DOT&E  published  a  MIDS  JTRS  VCD  report  in  March  2012. 
DOT&E  determined  that  testing  was  adequate  to  indicate 
that  the  MIDS  JTRS,  integrated  into  the  F/A-18E/F,  was 
now  operationally  effective  and  operationally  suitable. 

Major  deficiencies  discovered  during  the  2010  IOT&E, 
including  ineffective  Link  1 6  message  exchange  and  fine 
synchronization,  as  well  as  terminal  and  host  platform 
integrated  system  reliability,  have  been  resolved. 

•  The  Air  Force  completed  operational  testing  of  the 
MIDS  JTRS  core  terminal  integration  into  the  E-8C  Joint 
Surveillance  and  Target  Attack  Radar  System  (JSTARS) 
aircraft  in  September  2011.  DOT&E  assessed  that  MIDS 
JTRS  as  integrated  in  the  E-8  JSTARS  is  operationally 
effective  and  suitable  with  limitations. 

•  The  Navy  is  continuing  development  of  two  major  increments 
of  MIDS  JTRS  capability  that  will  improve  datalink 
performance:  Link  16  four-channel  Concurrent  Multi-Netting 
with  Concurrent  Retention  Receive  (CMN-4)  and  Tactical 
Targeting  Networking  Technology  (TTNT).  These  new 
capabilities  may  require  significant  hardware  and  software 
design  changes  to  the  MIDS  JTRS  core  terminal,  as  well  as 
modifications  to  host  platforms.  DOT&E  and  the  Navy  are 
formulating  the  test  strategies  for  these  new  MIDS  JTRS 
terminal  designs  and  their  integration  into  host  platforms. 

System 

•  When  integrated  into  a  host  platform,  MIDS  JTRS 
provides  Link  16  digital  datalink.  Link  16  digital  voice 
communications,  and  Tactical  Air  Navigation  (TACAN) 
capabilities,  plus  three  additional  programmable 
channels  capable  of  hosting  additional  JTRS  Software 
Communications  Architecture-compliant  waveforms  in  the 
2  to  2,000  Megahertz  radio  frequency  band.  In  addition, 
MIDS  JTRS  is  intended  to  provide  the  capability  for  Link  16 
enhanced  throughput  and  Link  16  frequency  re -mapping. 

•  Link  16  digital  datalink  is  a  joint  and  allied  secure  anti-jam 
high-speed  datalink  that  uses  standard  messages  to  exchange 
information  among  flight  or  battle-group  host  platforms  or 
between  combat  platforms  and  command  and  control  systems. 


Link  1 6  digital  voice  provides  host  platforms  with  a  secure 
anti-jam  voice  line-of-sight  communications  capability. 
TACAN  is  a  legacy  aircraft  navigation  system  used  in  many 
military  aircraft  with  air-to-air  as  well  as  air-to-ground  modes 
of  operation. 

•  The  system  includes  the  MIDS  terminals  and  the  host  platform 
components  and  interfaces  such  as  controls,  displays,  antennas, 
high-power  amplifiers,  and  any  radio  frequency  notch  filters. 

•  The  MIDS  JTRS  terminals  developed  to  conduct  Concurrent 
Multi-Net  Reception  are  intended  to  have  improved  digital 
receivers,  improved  buffering,  and  faster  processors  to  allow 
host  aircraft  to  receive  more  Link  16  messages  during  periods 
of  high  message  exchange  demand. 

•  The  Navy  intends  for  the  MIDS  JTRS  terminals  under 
development  to  utilize  the  multiband  capability  of  the  MIDS 
JTRS  to  provide  Link  16  and  TTNT  to  the  host  platform. 

The  Navy  intends  for  TTNT  to  provide  a  larger  throughput 
with  lower  latency,  thereby  enabling  faster  updates  of  precise 
information  than  Link  1 6  with  expanded  radio  frequency 
coverage.  The  Navy  intends  to  use  TTNT  as  one  of  the 
communications  enablers  for  the  Naval  Integrated  Fire 
Control  -  Counter  Air  capability. 

Mission 

U.S.  Services  and  many  allied  nations  will  deploy  aircraft,  ships, 
and  ground  units  equipped  with  MIDS  JTRS  in  order  to  provide 
military  commanders  with  the  ability  to  communicate  with  their 
forces  by  voice,  video,  and  data  during  all  aspects  of  military 
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operations.  MIDS  JTRS-equipped  units  are  intended  to  exchange 
information  including  air  and  surface  tracks,  identification,  host 
platform  fuel,  weapons,  mission  status,  engagement  orders, 
targeting  data,  and  engagement  results. 


Major  Contractors 

•  ViaSat,  Inc.  -  Carlsbad,  California 

•  Data  Link  Solutions  -  Wayne,  New  Jersey  and  Cedar  Rapids, 
Iowa 


Activity 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  completed  the  VCD  test  of  the  MIDS  JTRS,  as 
integrated  on  the  F/A-18E/F,  operating  from  the  Naval  Air 
Warfare  Center  China  Lake,  California,  and  during  deployed 
detachments  to  Naval  Air  Station  Fallon,  Nevada,  during 
December  2011. 

•  The  Air  Force’s  605th  Test  and  Evaluation  Squadron 
(TES)  conducted  a  Force  Development  Evaluation  of  the 
JSTARS  Communications  and  Network  Upgrade  Phase  1, 
which  includes  the  MIDS  JTRS  integration,  from  June  to 
September  2011.  This  evaluation  took  place  at  Patuxent  River 
Naval  Air  Station,  Maryland,  in  the  U.S.  Southern  Command 
Joint  Interagency  Task  Force  -  South  area  of  responsibility, 
and  at  the  JSTARS  Test  Force  and  Northrop  Grumman 
Corporation  facilities  in  Melbourne,  Florida.  All  testing  was 
conducted  in  accordance  with  DOT&E-approved  Test  and 
Evaluation  Master  Plans  and  operational  test  plans.  DOT&E 
published  a  MIDS  JTRS  VCD  report  in  March  2012. 

•  DOT&E  and  the  Navy  are  formulating  the  test  strategies  and 
revising  TEMPs  for  new  MIDS  JTRS  terminal  designs  and 
their  integration  into  host  platforms.  These  designs  include 
Link  16  four-channel  Concurrent  Multi-Netting  with  CMN-4 
and  TTNT. 

Assessment 

•  The  FY12  DOT&E  MIDS  JTRS  VCD  indicated  improved 
Link  16  message  exchange  compared  to  the  2010  IOT&E 
result.  DOT&E  determined  that  the  MIDS  JTRS,  as  integrated 
into  the  F/A-18E/F,  was  now  operationally  effective  and 
operationally  suitable. 

•  Testing  indicated  that  Link  16  messages  provided  situational 
awareness  of  friendly  force  positions  and  intentions.  These 
messages  were  consistently  and  accurately  exchanged  with 
100  percent  success  during  air-to-air  missions.  The  test 
data  also  indicated  a  99  percent  successful  ability  to  obtain 
Link  16  fine  synchronization  compared  to  the  IOT&E  result  of 
84  percent. 

•  The  test  data  indicated  that  terminal  and  host  platform 
integration  reliability  had  improved  relative  to  IOT&E. 

System  reliability  improved  to  21.6  hours  between  critical 


failures  compared  to  8.1  hours  during  the  IOT&E.  Operational 
availability  had  improved  to  99  percent  compared  to 
68  percent  during  the  IOT&E. 

•  The  built-in  test  false  alarm  rate  of  1  false  alarm  every 

5.9  flight  hours  failed  to  meet  the  threshold  requirement  of 
1  false  alarm  every  113  flight  hours. 

•  DOT&E  found  the  integration  of  the  MIDS  JTRS  core 
terminal  into  the  E-8C  JSTARS  operationally  effective 
and  operationally  suitable  with  limitations.  The  system 
was  effective  in  transmitting  Link  1 6  datalink  and  voice 
communications.  JSTARS  had  no  MIDS  JTRS  terminal 
failures  in  114.3  hours  accumulated  during  testing,  but 
terminal  operators  experienced  display  anomalies.  The 
JSTARS  Program  Office  plans  to  fix  the  display  anomalies  in 
the  next  upgrade  cycle. 

•  New  capabilities  such  as  Link  16  four-channel  Concurrent 
Multi-Netting  with  CMN-4  and  TTNT  may  require  significant 
hardware  and  software  design  changes  to  the  MIDS  JTRS  core 
terminal,  as  well  as  modifications  to  host  platforms  to  integrate 
the  new  TTNT  capability. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  and  the 
MIDS  Program  Office  made  satisfactory  progress  on  the 
previous  recommendations  related  to  the  correction  of 
deficiencies  identified  during  the  IOT&E. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Continue  software  design  efforts  to  reduce  the  incidence 
of  built-in  test  false  alarms  and  plan  a  maintenance 
demonstration  to  include  an  exhaustive  evaluation  of  failure 
diagnostics  prior  to  the  start  of  the  operational  test  of  the 
Link  16  Concurrent  Multi-Net  Reception  capability. 

2.  Adopt  a  Design  of  Experiments  approach  to  the  test 
strategies  and  include  a  reliability  growth  analysis/plan 
as  TEMP  revisions  occur  for  future  MIDS  JTRS 
capability  increments.  Conduct  a  thorough  review  of  the 
manufacturing  process  of  the  terminal  with  new  capabilities 
to  verify  the  new  production-representative  terminals  will 
be  stable  during  operational  testing. 
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Executive  Summary 

•  DoD  Public  Key  Infrastructure  (PKI)  Increment  2  provides 
authenticated  identity  management  via  password-protected 
Secret  Internet  Protocol  Routing  Network  (S1PRNET) 
tokens  to  enable  DoD  members  and  others  to  access  the 
S1PRNET  securely  and  encrypt  and  digitally  sign  e-mail.  Full 
implementation  will  enable  authorized  users  and  non-person 
entity  (NPE)-enabled  devices  (e.g.,  servers  and  workstations) 
to  access  restricted  websites  and  enroll  in  online  services. 

•  An  10T&E  in  2011  exposed  significant  logistics  deficiencies 
due  to  undefined  processes  for  procuring,  distributing,  and 
tracking  SIPRNET  tokens.  The  Defense  Manpower  Data 
Center,  which  currently  handles  the  common  access  card 
(CAC)  logistical  processes  on  the  Non-secure  Internet 
Protocol  Router  Network  (NIPRNET),  is  actively  working 
with  the  DoD  PKI  Program  Management  Office  (PMO)  to 
take  on  similar  responsibilities  for  the  SIPRNET  tokens. 

•  In  October  2011,  the  DoD  Chief  Information  Officer  (CIO) 
directed  the  military  Services  and  agencies  to  implement 
PKI  on  the  SIPRNET  and  deploy  tokens  DoD-wide  to 
approximately  500,000  users  by  December  31,  2012.  As  of 
mid-October  2012,  military  Services  and  agencies  had  issued 
approximately  78,300  tokens  to  users,  falling  far  short  of  the 
pace  needed  to  achieve  the  500,000  goal  by  end  of  year.  This 
status  reflects  growing  backlogs  in  registering,  enrolling,  and 
getting  SIPRNET  tokens  out  to  users,  suggesting  only  nominal 
improvements  toward  overall  SIPRNET  security. 

•  In  January  2012,  the  DoD  CIO  approved  full  fielding  of  PKI 
to  the  SIPRNET  and  tactical  environments  and  the  acquisition 
of  the  necessary  tokens,  card  readers,  and  software  to  support 
fielding.  The  DoD  CIO  called  for  an  operational  retest 

of  end-to-end  logistical  processes  to  address  outstanding 
suitability  deficiencies. 

•  In  September  2012,  the  Joint  Interoperability  Test  Command 
(J1TC)  conducted  an  operational  test  of  a  bulk  token 
formatting  and  issuance  capability  intended  to  improve 

the  speed  of  token  issuance.  J1TC  found  the  formatter  to 
be  effective  and  suitable,  fulfilling  its  intended  purpose 
by  providing  the  Registration  Authorities  the  ability  to 
simultaneously  complete  multiple  tasks.  Fifteen  SIPRNET 
token  bulk  formatters  are  currently  being  used  by  the  Services 
and  an  additional  35  units  are  being  procured. 

•  Capabilities  to  correct  logistic  shortfalls  with  token  inventory 
and  accounting  are  delayed  and  awaiting  operational  testing. 

A  previously  scheduled  FOT&E  slipped  from  3QFY12  until 
2QFY13  to  verify  correction  of  logistics  deficiencies  found 
during  the  IOT&E.  J1TC  intends  to  conduct  an  operational 
test  to  verify  whether  the  new  Inventory  Logistics  System 
(1LS)  can  successfully  track  tokens  by  serial  number  and 
location  throughout  their  lifecycle  from  ordering,  through 


Cross-Certification  (off-line) 


Internet  Environment 


ARA  -  Auto-key  Recovery  Agent 
CA  -  Certification  Authority 
CAC  -  Commom  Access  Card 
DEERS  -  Defense  Enrollment  Eligibility 
Reporting  System 

DMDC  -  Defense  Manpower  Data  Center 
DRM  -  Data  Recovery  Manager 
ECA  -  Enterprise  Certification  Authority 
GIG  -  Global  Information  Grid 
ID  -  Identification 


LRA  -  Local  Registration  Authority 
ORC  -  Operational  Research  Consultants,  Inc. 
PK  -  Public  Key 
RA  -  Registration  Authority 
RAPIDS  -  Real-Time  Automated  Personnel 
Identification  System 

SIPRNet  -  SECRET  Internet  Protocol  Router 
Network 

VO  -  Verifying  Official 


shipping,  issuing,  and  return.  Future  enhanced  reporting 
capabilities  to  improve  token  accounting  and  reporting  will 
not  be  available  for  operational  testing  until  1QFY14. 


System 

•  DoD  PKI  is  a  critical  enabling  technology  for  Information 
Assurance  (IA).  It  supports  the  secure  flow  of  information 
across  the  Global  Information  Grid  as  well  as  secure  local 
storage  of  information. 

•  DoD  PKI  provides  for  the  generation,  production,  distribution, 
control,  revocation,  recovery,  and  tracking  of  Public  Key 
certificates  and  their  corresponding  private  keys. 

•  DoD  PKI  is  comprised  of  commercial  off-the-shelf  hardware 
and  software,  and  other  applications  developed  by  the 
National  Security  Agency  (NSA). 

-  The  Defense  Enrollment  Eligibility  Reporting  System 
(DEERS)  and  the  Secret  DEERS  provide  the  personnel 
data  for  certificates  imprinted  on  NIPRNET  CACs  and 
SIPRNET  tokens  respectively. 

-  DoD  PKI  Certification  Authorities  for  the  NIPRNET 
and  SIPRNET  tokens  reside  in  the  Defense  Information 
Systems  Agency  (DISA)  Enterprise  Service  Centers 

in  Chambersburg,  Pennsylvania,  and  Oklahoma  City, 
Oklahoma. 

-  DISA  and  NSA  are  jointly  developing  DoD  PKI  in 
multiple  increments.  Increment  1  is  complete  and 
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deployed  on  the  NIPRNET.  Increment  2  is  being 
developed  and  deployed  in  three  spirals  on  the  SIPRNET 
and  NIPRNET  to  deliver  the  infrastructure,  PKI  services 
and  products,  and  logistical  support  for  Spiral  1  (tokens), 
Spiral  2  (tactical  and  austere  environments),  and  Spiral  3 
(NPE  PKI,  Federal  and  coalition  capabilities). 

•  The  NPE  development  effort  provides  certificates  for  devices, 
automated  and  manual  enrollment,  and  the  infrastructure 
means  for  credential  checking  to  insure  NPE-enabled  devices 
(e.g.,  domain  controllers,  web  servers,  and  workstations)  are 
authorized  to  exist  on  DoD  networks. 

Mission 

•  Military  operators,  communities  of  interest,  and  other 
authorized  users  will  use  DoD  PKI  to  securely  access,  process, 


store,  transport,  and  use  information,  applications,  and 
networks  regardless  of  technology,  organization,  or  location. 

•  Commanders  at  all  levels  will  use  DoD  PKI  to  provide 
authenticated  identity  management  via  personal  identification 
number-protected  CACs  or  SIPRNET  tokens  to  enable  DoD 
members,  coalition  partners,  and  others  to  access  restricted 
websites,  enroll  in  online  services,  and  encrypt  and  digitally 
sign  e-mail. 

Major  Contractors 

•  General  Dynamics  Information  Technology  -  Needham, 
Massachusetts  (Prime) 

•  90Meter  -  Newport  Beach,  California 

•  SafeNet  -  Belcamp,  Maryland 


Activity 

•  In  September  2011,  the  PKI  PMO  signed  a  Memorandum 
of  Agreement  with  the  Defense  Manpower  Data  Center  to 
develop  an  ILS  to  handle  SIPRNET  token  ordering,  shipping, 
and  distribution. 

•  In  October  2011,  the  DoD  CIO  directed  the  military  Services 
and  Agencies  to  implement  PKI  on  the  SIPRNET  and  deploy 
tokens  DoD-wide  to  all  users  by  December  31,  2012. 

•  The  JITC  and  PMO  executed  an  automated  failover  in 
December  2011.  The  failover  between  the  PKI  systems  in 
Chambersburg,  Pennsylvania,  and  Oklahoma  City,  Oklahoma, 
demonstrated  continuity  of  operations  after  the  planned 
failover. 

•  DOT&E  issued  the  PKI  Increment  1,  Spirals  1  and  2  IOT&E 
report  on  December  6,  2011. 

•  In  January  2012,  the  DoD  CIO  approved  full  fielding  of  PKI 
to  the  SIPRNET  and  tactical  environments  and  the  acquisition 
of  the  necessary  tokens,  card  readers,  and  software  to  support 
fielding.  The  DoD  CIO  called  for  an  operational  retest 

of  end-to-end  logistical  processes  to  address  outstanding 
suitability  deficiencies. 

•  The  PKI  PMO  continues  to  work  on  the  underlying  suitability 
problems,  focusing  on  the  ILS  and  Token  Management  System 
to  address  scalability.  However,  no  significant  PKI  capabilities 
were  delivered  and  operationally  tested  in  FY12.  Suitability 
retesting  is  ready  but  on  hold  due  to  errors  in  reconciling  token 
inventory  status  for  tokens  already  issued. 

•  Test  planning  and  preparations  are  ongoing  for  conducting  an 
operational  test  of  Increment  2,  Spiral  3  backend  capabilities 
in  December  2012. 

•  With  approval  from  the  DoD  CIO,  the  PKI  PMO  further 
delayed  the  NPE  capability  delivery  and  operational 
assessment  until  3QFY13  due  to  security  and  user  concerns 
that  the  proposed  capability  will  not  satisfy  current  mission 
requirements.  Security  mitigations  are  being  developed 
and  plans  for  testing  the  proposed  NPE  capability  are 
being  put  in  place. 


Assessment 

•  The  PKI  Increment  2,  Spirals  1  and  2  IOT&E  were  conducted 
in  4QFY11  in  accordance  with  a  DOT&E-approved  test  plan. 

•  The  DOT&E  report  in  December  2011  found  the  system  to 
be  operationally  effective  and  secure,  but  not  operationally 
suitable  with  logistical  improvements  needed  to  support  full 
deployment.  The  system  was  available,  reliable,  maintainable, 
and  relatively  easy  to  use.  However,  critical  logistical  support 
deficiencies  were  found  in  the  processes  for  distributing 

and  accounting  for  hardware  tokens,  token  readers,  and 
middleware.  Specific  logistics  deficiencies  found  include  the 
following: 

-  The  lack  of  a  robust  token  reliability  tracking  program  for 
ensuring  failure  rates  are  acceptable. 

-  A  centralized  token  inventory  system  for  ensuring  tokens 
are  procured  and  distributed  according  to  Service  and 
agency  demands  and  for  tracking  tokens  that  are  reissued 
to  different  users. 

-  The  need  for  more  customized  searching  and  reporting 
through  the  Token  Management  System  portal  interface 
to  enable  Services  and  Agencies  to  better  report  on  token 
issuance  status. 

-  The  lack  of  bulk  formatting  and  issuance  capabilities  to 
improve  token  distribution  time  and  reduce  manpower 
costs. 

•  While  logistics  improvements  have  been  made  since  the 
IOT&E  report,  to  include  delivery  of  a  bulk  formatting  and 
issuance  capability,  a  final  tested  robust  logistics  process  for 
token  procuring,  distributing,  tracking,  and  reporting  is  not 
in  place.  Delays  in  the  delivery  of  the  centralized  logistics 
inventory  system  coupled  with  the  Services’  need  to  meet 
the  DoD  CIO’s  SIPRNET  token  deployment  directive  are 
hampering  the  transition  from  previous  operations. 

•  There  are  over  530,000  SIPRNET  tokens  distributed  to  the 
Services  and  Agencies  but  only  78,300  tokens  in  use  as  of 
mid-October  2012.  This  status  reflects  the  significant  backlog 
in  registering,  enrolling,  and  deploying  SIPRNET  tokens.  The 
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Service  and  agency  registration  authorities  responsible  for 
issuing  tokens  currently  do  not  have  the  manpower  needed  to 
support  the  FY13  NPE  operational  test  activities.  Additionally, 
stakeholders  do  not  have  enough  confidence  in  the  NPE 
capability  because  it  has  not  been  adequately  demonstrated 
via  developmental  testing.  Ongoing  missions  and  operations 
are  precluding  the  Services  and  agencies  from  being  able  to 
commit  critical  devices  to  operational  assessments  and  testing. 

•  Overall,  the  PKI  system  and  technical  capabilities  are  sound, 
but  the  SIPRNET  standard  operating  procedures,  training, 
logistical  support,  and  lifecycle  sustainment  lack  maturity. 
While  procedures  for  middleware  and  card  reader  distribution 
and  support  have  improved,  the  new  process  supporting  the 
token  1LS  has  not  yet  been  demonstrated.  Additional  duties 
are  being  defined  within  the  PKI  program  for  Service  and 
agency  token  warehouse  and  issuance  site  managers  to  ensure 
that  tokens  are  properly  tracked  by  the  new  system.  With 
more  than  100  issuance  sites,  there  is  a  significant  risk  that 
the  ILS  process  will  not  be  uniformly  adopted  in  a  timely 
manner,  which  is  necessary  to  provide  a  complete  and  accurate 
inventory  status  and  to  enable  production  of  tokens  based  on 
issuance  site  requirements. 

•  The  NPE  capability  delays  continue  to  plague  the  Program 
Office.  The  NPE  delivery  is  now  amassed  into  the 
Increment  2,  Spiral  3  set  of  capabilities,  which  also  include 
enhancements  to  the  existing  SIPRNET  and  NIPRNET 
infrastructures.  While  the  NPE  Release  3  provides  some 
automation  improvements,  it  adds  a  substantial  manpower 
support  requirement  for  the  Services  to  accomplish  the 
large  volume  of  DoD  NPE  devices,  especially  given  the 
loose  DoD  guidance  stating  that  all  devices  require  medium 
assurance  certificates.  The  registration  authorities  and  system 
administrators  ultimately  responsible  for  approving  and 
enrolling  devices  into  the  system  have  major  concerns  with  the 
system  and  the  workload  it  will  entail.  Although  the  PKI  PMO 
has  attempted  to  refine  guidance  to  enable  NPE  certificate 
deployment  to  be  prioritized  based  on  risk  that  a  device  can  be 
compromised  by  information  attacks,  the  community  has  not 
established  a  clear  path  forward  for  deploying  NPE  certificates 
across  DoD  devices. 

•  The  developmental  test  program  is  inadequate  to  support 
integrated  test  planning  efforts.  Processes  and  procedures 


directed  in  both  the  Test  and  Evaluation  Master  Plan  and 
System  Engineering  Plan  have  not  been  implemented,  which 
has  resulted  in  limited  visibility  into  actual  performance  of 
the  system.  Better  coordination  between  the  test  teams,  and 
improved  test  planning  and  reporting  are  required  to  support 
operational  test  readiness  assessments. 

•  Overly  aggressive  testing  event  dates  continue  to  waste  critical 
user  and  test  resources  by  forcing  the  assessment  of  PKI 
capabilities  that  are  not  ready  to  be  assessed. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  PKI  PMO 
satisfactorily  addressed  five  of  seven  recommendations  from 
the  FY11  Annual  Report  for  Increment  2,  Spirals  1  and  2.  The 
recommendations  concerning  correction  of  token  testing  and 
scheduling  deficiencies  remain. 

•  FY12  Recommendations. 

1 .  The  PKI  program  needs  to  improve  coordination  with 
the  stakeholders,  provide  better  capability  definition,  and 
recognize  schedule  impossibilities  early  to  provide  sound 
acquisition  management  for  testing  and  delivering  the  PKI 
capabilities  for  the  DoD. 

2.  The  PKI  program  needs  to  adhere  to  both  the  System 
Engineering  Plan  and  Test  and  Evaluation  Master  Plan, 
which  provide  specific  direction  on  how  the  development 
and  testing  of  the  PKI  capabilities  should  be  executed. 

3.  The  PKI  acquisition  community  needs  clear  guidance  on 
the  intended  NPE  devices.  Any  directives  for  issuing  NPE 
certificates  must  take  into  consideration  Service  and  agency 
manpower  and  resource  constraints.  If  such  guidance  is 
not  timely,  the  PKI  acquisition  program  baseline  should  be 
restructured  such  that  tests  are  driven  based  on  capability 
maturity,  readiness,  and  mission  need  and  not  to  satisfy 
program  schedule  requirements. 

4.  The  PMO  should  work  to  establish  a  more  realistic  timeline 
for  PKI  development,  delivery,  and  capability  testing  that 
better  supports  milestone  decisions.  The  program  must 
better  manage  expectations  of  those  with  PKI  equities  by 
avoiding  recurring  schedule  slips  caused  by  capability 
delivery  delays. 
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Network  Integration  Evaluation  (NIE) 


In  FY12,  the  Amy  continued  execution  of  a  series  of  Network 
Integration  Evaluations  (NIEs),  which  began  in  FY11  with 
NIE  11.2.  The  purpose  of  the  NIEs  is  to  provide  a  venue 
for  operational  testing  of  Army  acquisition  programs,  with  a 
particular  focus  on  the  integrated  testing  of  programs  related  to 
tactical  mission  command  networks.  Additionally,  the  NIEs  are 
intended  to  serve  as  a  venue  for  evaluating  emerging  capabilities 
that  are  not  formal  acquisition  programs.  This  fiscal  year  the 
Amy  executed  two  NIEs,  both  at  Fort  Bliss,  Texas,  and  White 
Sands  Missile  Range,  New  Mexico.  NIE  12.1  was  conducted 
from  October  through  November  2011  and  NIE  12.2  was 
conducted  from  April  through  May  2012. 

The  intended  objective  of  the  NIE  to  test  and  evaluate  network 
components  together  in  a  combined  event  is  sound.  The  NIE 
events  should  allow  for  a  more  comprehensive  evaluation  of 
an  integrated  mission  command  network,  instead  of  piecemeal 
evaluations  of  individual  network  components.  Conducting 
NIEs  two  times  a  year  creates  an  opportunity  for  event-driven 
operational  testing  as  opposed  to  schedule-driven  testing. 

For  example,  if  a  system  was  not  developmentally  ready  to 
enter  operational  testing  at  one  NIE  event,  it  would  have  the 
opportunity  to  enter  testing  in  a  subsequent  NIE  event. 

The  Army  intends  to  conduct  NIE  events  approximately  every  six 
months  for  the  foreseeable  future. 

NIE  12.1. 

During  NIE  12.1,  the  Joint  Tactical  Radio  System  (JTRS) 
Flandheld,  Manpack,  and  Small  Form  Fit  (F1MS)  Rifleman 
Radio  underwent  an  IOT&E.  The  Army  also  executed  an 
operational  assessment  of  the  AN/PRC-1 17G  radio.  (Individual 
articles  providing  assessments  of  the  Rifleman  Radio  and  the 
AN/PRC-1 17G  can  be  found  later  in  this  Annual  Report). 


During  NIE  12.1,  the  Army  conducted  evaluations  of  47 
additional  systems  in  various  stages  of  development.  These 
systems,  termed  by  the  Army  as  “systems  under  evaluation” 
(SUEs),  are  not  acquisition  programs  of  record,  but  rather 
systems  that  may  offer  value  for  future  development. 

NIE  12.2 

During  NIE  12.2,  the  Army  conducted  a  Multi-Service 
Operational  Test  and  Evaluation  (MOT&E)  for  the  JTRS 
HMS  Manpack  radio  and  IOT&Es  for  Warfighter  Information 
Network  -  Tactical  ( WIN-T)  Increment  2  and  JTRS  Enterprise 
Network  Manager.  (Individual  articles  on  these  programs  are 
provided  later  in  this  Annual  Report.)  The  Army  also  conducted 
assessments  of  36  SUEs. 


NIE  ASSESSMENT 


Overall,  the  Army’s  execution  of  the  NIEs  has  shown  steady 
improvement  since  NIE  11.2,  which  was  reported  on  in  the  FY11 
Annual  Report.  The  Army  has  incorporated  lessons  learned  from 
previous  events.  This  was  reflected  in  NIE  12.2,  which  was  the 
best  planned  and  executed  NIE  of  the  three  conducted  to  date. 

Operational  Scenarios  and  Test  Design.  The  Brigade 
Modernization  Command,  in  conjunction  with  the  Army  Test  and 
Evaluation  Command’s  Operational  Test  Command,  continues 
to  develop  realistic,  well  designed  operational  scenarios  for  use 
during  NIEs.  NIE  12.2  contained  the  most  challenging  mission 
set  to  date,  including  Combined  Arms  Maneuver  for  the  first  time 
in  any  significant  way.  NIE  12.2  stressed  brigade  and  battalion 
command  and  control  to  a  much  greater  degree  than  did  previous 
NIEs.  Unlike  in  previous  NIEs,  all  Tactical  Operations  Centers 


and  command  posts  in  NIE  12.2  displaced  multiple  times. 
Additionally,  as  part  of  the  WIN-T  Increment  2  testing,  NIE  12.2 
included  units  not  located  at  Fort  Bliss  or  White  Sands  Missile 
Range,  with  the  101st  Airborne  Division  operating  from  Fort 
Campbell,  Kentucky,  serving  as  the  division  headquarters. 

In  contrast  to  NIE  12.2,  NIE  12.1  focused  on  Wide  Area  Security 
missions  executed  predominately  at  the  company  level.  Brigade 
and  battalion  tactical  operations  centers  and  company  command 
posts  operated  from  fixed  sites  and  were  dependent  upon  a  fixed 
aerial  tier  of  100-foot  towers  and  aerostats  in  order  to  establish 
network  connectivity. 

Threat  Operations.  The  Army  continues  to  improve  threat 
operations  during  NIEs.  NIE  12.2  was  the  first  NIE  in  which 
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threat  information  operations,  such  as  electronic  warfare  and 
computer  network  operations,  were  fully  integrated  into  the  threat 
commander’s  scheme  of  maneuver.  An  aggressive,  adaptive 
threat  intent  on  winning  the  battle  is  an  essential  component  of 
good  operational  testing.  The  Army  should  continue  to  ensure 
that  future  NIEs  contain  a  robust  threat  force,  to  include  threat 
information  warfare  capabilities. 

Schedule-Driven  Programs.  The  Army  remains  schedule-driven 
to  meet  NIE  objectives,  vice  pursuing  an  event-driven  schedule 
appropriate  to  acquisition  system  development. 

In  NIE  12.1  the  Army  proceeded  to  the  Rifleman  Radio  IOT&E 
despite  unfinished  developmental  testing.  Developmental  testing 
conducted  prior  to  the  IOT&E  indicated  that  the  Rifleman  Radio 
was  demonstrating  reliability  of  roughly  one-half  its  intended 
requirement  (277  hours  of  Mean  Time  Between  Essential 
Function  Failure  (MTBEFF)  compared  to  a  requirement 
of  477  hours).  Predictably,  Rifleman  Radio’s  reliability 
demonstrated  during  the  IOT&E  fell  far  short  of  the  requirement, 
demonstrating  a  161-hour  MTBEFF. 

In  NIE  12.2,  the  Army  proceeded  with  the  Manpack  radio 
MOT&E  despite  developmental  testing  results  indicating 
that  the  Manpack  radio  Single  Channel  Ground  and  Airborne 
Radio  System  (SINCGARS)  waveform  performance  was  not 
satisfactory  and  that  the  Manpack  radio  was  falling  short  of  its 
reliability  requirements.  The  OSD’s  Deputy  Assistant  Secretary 
of  Defense  for  Developmental  Test  and  Evaluation  in  its 
Assessment  of  Operational  Test  Readiness  (AOTR)  identified 
these  developmental  testing  deficiencies  prior  to  the  start  of  the 
MOT&E.  The  Manpack  radio  MOT&E  results  confirmed  both 
the  SINCGARS  waveform  deficiency  and  reliability  shortfall 
previously  identified  in  developmental  testing.  Furthermore, 
Manpack  radio  vehicle  integration  was  poorly  executed  and 
not  adequately  tested  prior  to  the  MOT&E,  resulting  in  poor 
vehicular-mounted  radio  performance  in  the  MOT&E.  The 
AOTR  also  identified  insufficient  developmental  testing  to  assess 
vehicle  integration  prior  to  the  MOT&E. 

According  to  system  development  best  practices,  the  Army 
should  not  proceed  to  IOT&E  with  a  system  until  it  has 
completed  developmental  testing  and  the  program  has 


corrected  any  identified  problems.  Otherwise,  the  Army  runs 
the  risk  of  conducting  costly  operational  tests  that  only  serve 
to  confirm  developmental  testing  conclusions  about  poor 
system  performance,  without  affording  the  Program  Office  the 
opportunity  to  fix  system  shortfalls. 

Too  Many  Systems.  The  Army  continues  to  insert  a  large 
number  of  immature  systems  into  NIEs.  The  25  SUEs 
contained  in  the  NIE  11.2  stressed  the  Army’s  evaluation 
capacity,  indicating  a  need  to  reduce  the  number  of  SUEs  in 
later  NIEs.  Nevertheless,  NIE  12.1  contained  47  SUEs  and 
NIE  12.2  included  36  SUEs.  Too  many  immature  systems  in 
the  NIE  challenges  the  Army’s  capacity  to  employ  appropriate 
instrumentation,  collect  relevant  data,  and  conduct  full  and 
adequate  assessments,  detracting  from  the  Army’s  capability 
to  perform  focused  evaluations.  Having  this  large  number  of 
SUEs  in  the  NIE  stresses  the  test  unit’s  capacity  to  adequately 
train  operators  on  the  new  systems.  In  addition,  it  strains  the 
unit’s  ability  to  understand  the  systems’  intended  concepts  of 
operations  and  to  integrate  the  systems  into  unit  operations. 

These  limitations  make  it  challenging  to  thoroughly  examine  the 
capabilities  of  each  SUE.  Finally,  much  of  the  NIEs’  overall  cost 
can  be  attributed  to  the  inclusion  of  these  SUEs.  Whether  the 
knowledge  gained  of  the  SUEs  justifies  the  overall  cost  is  unclear. 

Logistics.  NIEs  are  still  not  replicating  realistic  battlefield 
maintenance  and  logistical  support  operations  for  systems 
under  test.  Field  Service  Representative  (FSR)  support  plans, 
maintenance  and  repair  parts  stockage,  and  the  quantity  and 
management  of  system  spares,  do  not  accurately  reflect  what  a 
unit  will  observe  upon  fielding.  For  example,  the  JTRS  HMS 
Manpack  System  Support  Package  provided  by  the  Program 
Office  for  use  in  the  MOT&E  did  not  accurately  reflect  expected 
system  support  after  fielding.  The  density  of  contract  FSRs  and 
spare  radios  far  exceeded  what  is  likely  to  be  the  case  within 
fielded  brigades.  Easy  access  and  over  reliance  on  FSR  support 
resulted  in  the  test  unit  not  having  to  realistically  execute  its 
field-level  maintenance  actions.  Failure  to  accurately  replicate 
“real  world”  maintenance  and  logistics  support  cause  operational 
availability  rates  and  ease  of  maintenance  to  be  overestimated  in 
NIEs. 
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AN/PRC-1 17G 


Executive  Summary 

•  The  Army  intends  tactical  units  to  employ  the  AN/PRC-1 17G 
as  a  data  radio.  The  radio  will  be  used  as  an  interim 
commercial  off-the-shelf  (COTS)  solution  until  the  Multi-Tier 
Networking  Vehicle  Radio  (MNVR)  is  developed  and  fielded. 
The  MNVR  is  intended  to  replace  the  cancelled  Joint  Tactical 
Radio  System  Ground  Mobile  Radio  (JTRS  GMR)  program. 

•  The  AN/PRC- 11 7G  radio  is  not  a  Program  of  Record. 

As  a  result,  there  is  no  acquisition  strategy,  documented 
requirements,  or  Test  and  Evaluation  Master  Plan.  The  Army 
has  used  an  existing  General  Services  Administration  contract 
to  purchase  the  AN/PRC-1 17G. 

•  The  Army  Test  and  Evaluation  Command  conducted  an 
operational  assessment  of  the  AN/PRC-1 17G  from  October 
through  November  2011  at  Fort  Bliss,  Texas,  and  White  Sands 
Missile  Range,  New  Mexico,  in  conjunction  with  the  Army’s 
Network  Integration  Evaluation  (NIE)  12.1. 

•  The  AN/PRC- 1 1 7G  performed  satisfactorily  in  transmitting 
digital  Position  Location  Information.  However,  full 
AN/PRC-117G  capability  was  not  exercised  or  evaluated. 
AN/PRC-117G  demonstrated  satisfactory  reliability  and 
interoperability  with  the  Rifleman  Radio  and  its  corresponding 
Soldier  Radio  Waveform  (SRW)  network. 

System 

•  The  AN/PRC- 11 7G  radio  is  a  single  channel  voice  and 
data  radio  that  is  capable  of  operating  in  a  frequency  range 
of  30  Megahertz  to  2  Gigahertz.  It  can  be  configured  for 
manpack,  vehicular-mounted,  or  base  station  operations. 

The  primary  AN/PRC-1 17G  waveform  is  the  Advanced 
Networking  Wideband  Waveform  (ANW2),  which  is  a  Harris 
Corporation  proprietary  waveform.  The  AN/PRC-1 17G 

is  capable  of  simultaneously  transmitting  both  Voice  over 


Internet  Protocol  (VoIP)  and  digital  data  on  a  single  channel. 
Digital  data  include  file  transfers,  chat,  streaming  video,  and 
position  location  reports.  ANW2  allows  units  to  use  internet 
protocol  routing  to  transmit  medium  to  high  bandwidth 
data  traffic  over  tactical  Very  High  Frequency,  Ultra  High 
Frequency,  and  L-band  radio  networks. 

•  The  AN/PRC- 11 7G  radio  is  not  a  Program  of  Record. 

As  a  result,  there  is  no  acquisition  strategy,  documented 
requirements,  or  Test  and  Evaluation  Master  Plan. 

Mission 

The  Army  intends  for  tactical  units  to  employ  the  AN/PRC-1 17G 
as  a  data  radio.  AN/PRC-1 17G  will  be  an  interim  COTS  solution 
until  the  MNVR  is  developed  and  fielded.  The  MNVR  is 
intended  to  replace  the  cancelled  JTRS  GMR  program. 

Major  Contractor 

Harris  Corporation  -  Rochester,  New  York 


Activity 

•  The  Army  is  purchasing  the  AN/PRC-1 17G  as  a  COTS 
item  to  fill  a  capability  gap  for  a  tactical  digital  radio.  With 
the  October  20 1 1  cancellation  of  JTRS  GMR,  the  Army 
sought  an  interim  solution  to  fill  Brigade  Combat  Teams  as 
a  part  of  Capability  Set  13.  The  Army  has  used  an  existing 
General  Services  Administration  contract  to  purchase  the 
AN/PRC- 117G. 

•  The  Army  Test  and  Evaluation  Command  conducted  an 
operational  assessment  of  the  AN/PRC-1 17G  from  October 
through  November  2011  at  Fort  Bliss,  Texas,  and  White  Sands 
Missile  Range,  New  Mexico,  in  conjunction  with  the  Army’s 
NIE  12.1.  During  this  event,  a  cavalry  squadron  equipped 
with  Mine  Resistant  Ambush  Protected  vehicles  executed 


typical  cavalry  missions  in  an  Afghanistan-like  scenario.  A 
total  of  26  AN/PRC-1 17Gs  were  used.  These  radios  ran  both 
the  AN  W2  and  SRW. 

•  DOT&E  has  placed  the  AN/PRC-1 17G  on  oversight. 

Assessment 

•  During  the  NIE  event,  the  AN/PRC- 1 1 7G  performed 
satisfactorily  in  transmitting  digital  data.  However,  full 
AN/PRC-1 17G  capability  was  not  exercised  or  evaluated. 

The  test  unit  only  employed  the  AN/PRC-1 17G  to  transmit 
limited  types  of  digital  Joint  Variable  Message  Format 
(JVMF)  messages.  The  vast  majority  of  the  digital  traffic 
was  automated  position  location  reports,  with  a  small  number 
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of  other  JVMF  messages  pertaining  to  mortar  fire  missions. 
The  unit  did  not  employ  the  radio’s  voice  capability,  except 
in  the  case  of  the  Rifleman  Radio-equipped  platoon.  The 
AN/PRC- 117G’s  capabilities  for  file  transfer,  transmitting 
streaming  video,  and  chat  were  not  employed  in  NIE  12.1, 
as  the  Army  did  not  give  the  test  unit  the  capability  or 
opportunity  to  do  so. 

•  The  SRW  successfully  ran  on  several  AN/PRC-1 17Gs,  which 
allowed  the  Rifleman  Radio-equipped  platoon  to  transmit  to 
the  AN/PRC- 11 7G  the  position  location  reports  generated  by 
Rifleman  Radio-equipped  Soldiers.  From  the  AN/PRC- 1 17G, 
these  position  locations  crossed  over  to  the  ANW2  in  the 
platoon  leader’s  vehicle,  enabling  these  position  locations  to 
be  displayed  at  the  troop  and  squadron  level.  This  troop  also 
successfully  used  the  AN/PRC- 117G  running  SRW  for  VoIP 
with  Rifleman  Radio-equipped  platoon  members. 


•  The  AN/PRC- 1 1 7G  demonstrated  a  point  estimate  Mean  Time 
Between  Essential  Function  Failure  (MTBEFF)  of  511  hours, 
which  exceeds  the  JTRS  GMR  single  channel  requirement  of 
477  hours  MTBEFF.  This  result  is  over  five  times  greater  than 
the  JTRS  GMR  reliability  demonstrated  in  NIE  11.2. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendation. 

1 .  The  Army  should  conduct  sufficient  government 

developmental  and  operational  testing  to  fully  characterize 
system  performance.  A  future  operational  assessment 
should  be  executed  in  which  the  test  unit  employs  the 
system  to  its  full  capability,  including  streaming  video,  file 
transfers,  chat,  and  VoIP. 
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Apache  Block  III  (AB3)  Upgrade 


Executive  Summary 

•  The  Army  conducted  the  Apache  Block  III  (AB3)  IOT&E 
at  the  National  Training  Center  in  Fort  Irwin,  California, 
March  through  April  2012.  The  test  included  training, 
force-on-force  missions,  live  fire  of  all  weapon  systems,  and 
threat  penetration  testing  of  AB3  computer  networks.  The 
IOT&E  was  preceded  by  four  years  of  developmental  testing 
that  included  analysis,  modeling  and  simulation,  component 
qualification  testing,  testing  in  extreme  environments, 
system-level  flight  testing,  weapons  qualifications,  and  live 
fire  testing. 

•  The  Army  completed  LFT&E  of  AB3  flight  critical  and  force 
protection  components  followed  by  vulnerability  analyses  to 
evaluate  aircraft  ballistic  survivability. 

•  In  August  2012,  DOT&E  submitted  a  Combined  Initial 
Operational  Test  and  Evaluation  and  Live  Fire  Test  and 
Evaluation  report  to  Congress  evaluating: 

-  AB3  as  operationally  effective.  It  has  improved  flight 
performance  compared  to  legacy  Apache  Block  II  (AB2) 
aircraft.  When  aided  by  real-time  unmanned  aircraft 
system  (UAS)  video  containing  actionable  intelligence, 
AB3  teams  demonstrated  target  acquisition  ranges,  Hellfire 
missile  engagement  ranges,  and  mission  success  rates 
greater  than  the  legacy  AB2  aircraft. 

-  AB3  as  operationally  suitable.  The  aircraft  exceeded 
reliability  thresholds  with  statistical  confidence  and  met 
all  current  maintainability  requirements.  Overall,  flight 
safety  is  enhanced  by  the  aircraft’s  ability  to  operate  with 
increased  power  margins. 

-  AB3  as  at  least  as  survivable  as  legacy  AB2  aircraft.  New 
AB3  subsystems  met  survivability  requirements  and 
demonstrated  ballistic  tolerance  similar  to  legacy  AB2 
aircraft. 

System 

•  The  AB3  is  a  modernized  version  of  the  AH-64D  Attack 
Helicopter  with  which  the  Army  intends  to  sustain  the  Apache 
fleet  through  the  year  2040.  The  Army  intends  to  organize 
the  AB3  in  Attack/Reconnaissance  Battalions  assigned  to 

the  Combat  Aviation  Brigades.  Each  Battalion  will  have 
24  aircraft. 

•  The  Army  acquisition  objective  is  to  procure  690  AB3 
aircraft:  634  remanufactured  and  56  new  build  aircraft. 
Remanufactured  and  new  build  AB3  aircraft  are  built  on  the 
same  assembly  line  in  Mesa,  Arizona,  and  are  essentially  the 
same  aircraft. 

•  The  AB3  aircraft  capability  improvements  include: 

-  Levels  2  through  4  UAS  control  by  the  AB3  aircrew 

■  Level  2  receives  and  displays  UAS  data/video  feed 

■  Level  3  receives  and  displays  UAS  data/video  feed  and 
enables  the  aircrew  to  control  the  UAS  payload  (sensor) 


■  Level  4  receives  and  displays  UAS  data/video  feed  and 
enables  the  aircrew  to  control  the  flight  path  of  the  UAS 
and  the  payload 

-  Improved  Radar  Electronic  Unit  to  provide  radio  frequency 
interferometer  passive  ranging,  extended  fire  control  radar 
range,  and  maritime  targeting  capability 

-  Improved  aircraft  performance  with  70 ID  engines, 
composite  main  rotor  blades,  weight  reduction  through 
processor  and  avionic  upgrades,  and  an  improved  rotor 
drive  system 

-  Enhanced  survivability  with  integrated  aircraft 
survivability  equipment  and  additional  crew  and  avionic 
armoring 

-  Enhanced  communication  capability,  which  includes 
satellite  communication,  Link  1 6  datalink,  and  an 
integrated  communication  suite  to  meet  global  air  traffic 
management  requirements 

-  Improved  reliability  and  maintainability  using  embedded 
system-level  diagnostics,  improved  electronic  technical 
manuals,  and  reduced  obsolescence 

Mission 

The  Attack/Reconnaissance  Battalions  assigned  to  the  Combat 

Aviation  Brigade  will  employ  the  AB3  to  conduct  the  following 

types  of  missions: 

•  Attack 

•  Movement  to  contact 

•  Reconnaissance 

•  Security 

Major  Contractors 

•  Aircraft:  The  Boeing  Company  Integrated  Defense 

Systems  -  Mesa,  Arizona 

•  Sensors  and  UAS  datalink:  Longbow  Limited  -  Orlando, 

Florida,  and  Baltimore,  Maryland 
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Activity 

•  The  Army  conducted  the  AB3  IOT&E  in  accordance  with  the 
DOT&E-approved  test  plan  from  March  16,  2012,  through 
April  13,  2012,  at  the  National  Training  Center  in  Fort  Irwin, 
California.  The  IOT&E  consisted  of  force-on- force  missions 
with  a  dedicated  opposing  force;  live  fire  of  all  weapon 
systems;  and  threat  penetration  testing  of  AB3  computer 
networks. 

•  Prior  to  the  IOT&E,  the  test  unit  completed  three  weeks 
of  classroom,  simulator,  and  flight  training  at  the  Boeing 
facility  in  Mesa,  Arizona.  Legacy  AB2  aircraft  and  aircrews 
conducted  home-station  training  prior  to  deploying  to  the 
National  Training  Center. 

•  During  IOT&E,  five  AB3  aircraft  flew  367  flight  hours.  The 
five  AB3  aircraft  and  five  legacy  AB2  aircraft  conducted 

28  force-on-force  missions  under  varying  conditions.  The 
conditions  were  selected  using  Design  of  Experiments 
methodology  with  four  factors:  aircraft  type  (AB3  or  AB2), 
mission  type  (reconnaissance  or  attack),  UAS  support  (with 
or  without),  and  light  level  (day  or  night).  A  single  Gray 
Eagle  UAS  with  associated  flight  crew  and  personnel  provided 
mission  support  to  the  IOT&E  from  Edwards  AFB,  California. 

•  The  Army  conducted  ballistic  testing  of 
production-representative  AB3-improved  drive  system 
components  and  composite  main  rotor  blades  (CMRBs) 
from  May  through  July  2011  at  Aberdeen  Proving  Ground, 
Maryland. 

•  The  Army  conducted  ballistic  shots  against  20  crew  armor 
panels  on  December  7,  2010,  to  verify  performance  against  the 
key  performance  parameter  force  protection  threat. 

•  At  the  conclusion  of  ballistic  testing,  the  Army  completed 
a  system-level  vulnerability  analysis  using  a  modeling  and 
simulation  suite  that  models  target-threat  interaction  for  direct 
fire  and  small  projectiles  on  air  and  ground  systems. 

•  The  Air  Force  approved  the  Mission  Design  Series  request  to 
designate  the  AB3  as  the  AH-64E  on  September  6,  2012. 

•  In  August  2012,  DOT&E  submitted  a  Combined  Initial 
Operational  Test  and  Evaluation  and  Live  Fire  Test  and 
Evaluation  report  to  Congress. 

•  The  USD(AT&L)  conducted  a  Defense  Acquisition  Board  on 
August  16,  2012.  The  key  decisions  include  the  following. 

-  Approve  full-rate  production  for  the  Apache  Block  IIIA 
(AB3A)  Remanufacture  program. 

-  Authorize  the  procurement  of  up  to  12  low-rate  initial 
production  aircraft  in  FY13  for  the  Apache  Block  IIIB 
(AB3B)  New  Build  program. 

-  The  Army  will  fully  fund  the  AB3A  Remanufacture 
program  to  the  Director  of  Cost  Assessment  and  Program 
Evaluation-approved  independent  cost  estimate. 

-  Upon  approval  of  the  AB3  A  Remanufacture  Acquisition 
Program  Baseline,  the  AB3A  Remanufacture  and  AB3B 
New  Build  programs  will  be  designated  as  Acquisition 
Category  1C  programs. 


Assessment 

•  The  AB3  is  operationally  effective.  It  has  improved  flight 
performance  compared  to  legacy  AB2  aircraft,  and  when  aided 
by  real-time  UAS  video  containing  actionable  intelligence, 
AB3  teams  demonstrated  target  acquisition  ranges,  Hellfire 
engagement  ranges,  and  greater  mission  success  rates  than 
legacy  AB2  aircraft  teams. 

•  AB3  crews  were  consistently  able  to  establish  a  datalink  with 
Gray  Eagle  to  receive  UAS  video.  Crews  had  some  difficulty 
establishing  and  maintaining  control  of  the  Gray  Eagle  sensor. 

•  There  is  a  pilot  confidence  concern  with  the  AB3  transmission 
design.  It  has  a  single  tail  rotor  output  pinion  that  provides 
power  for  the  tail  rotor,  hydraulic  pump,  and  electric  generator. 
A  failure  of  this  one  pinion  could  be  catastrophic  for  the 
aircrew,  as  this  would  result  in  the  simultaneous  loss  of  the  tail 
rotor,  electric  generator,  and  hydraulic  power. 

•  AB3  is  operationally  suitable.  The  helicopter  exceeded  its 
reliability  thresholds  with  statistical  confidence  and  met  all 
current  maintainability  requirements.  The  redesigned  AB3 
helmet  offers  improved  comfort  and  performance  compared  to 
the  legacy  helmet.  Overall,  flight  safety  is  enhanced  by  AB3's 
increased  power  margins  relative  to  legacy  AB2  aircraft. 

•  The  AB3  is  at  least  as  survivable  as  the  legacy  AB2  aircraft. 
New  AB3  subsystems  met  survivability  requirements  and 
demonstrated  ballistic  tolerance  similar  to  legacy  AB2  aircraft. 

•  Infrared  countermeasures  provide  protection  against  most 
man-portable  rocket  system  threats,  but  the  Army  should 
improve  the  laser  and  radar  warning  systems.  Consistent  with 
other  DOT&E  evaluations,  the  APR-39A(V)4  radar  warning 
receiver  was  not  effective  during  IOT&E.  Radar  warning 
receiver  false  alarms  were  so  pervasive  during  the  IOT&E  that 
the  pilots  ignored  or  turned  off  all  countermeasure  warning 
systems. 

•  The  AB3  is  vulnerable  to  computer  network  attack.  An 
Army  threat  computer  network  operations  team  conducted 
limited  penetration  testing  of  AB3  computer  networks.  The 
threat  team  considered  three  attack  vectors  to  gain  access 
to  the  AB3  networked  systems:  the  Blue  Force  Tracker,  the 
Aviation  Mission  Planning  System,  and  aircraft  maintenance 
ports.  Threat  team  activities  were  limited  to  computer  network 
scanning  (passive  and  active)  while  the  AB3  aircraft  were  on 
the  ground,  so  as  not  to  affect  flight  operations.  The  team  was 
successful  in  gaining  access  to  AB3  systems. 

•  The  CMRB  has  very  low  vulnerability  to  most  small  arms 
threats.  However,  during  another  test,  a  larger  threat  impacted 
the  blade  spar  and  removed  a  substantial  portion  of  the 
spar’s  cross-sectional  area.  The  blade  completed  30  minutes 
of  operation,  despite  a  loss  of  structural  stiffness.  While 
spinning,  the  centrifugal  forces  kept  the  blade  straight,  but 

as  the  blade  rotation  slowed  after  the  shot,  the  blade  folded 
downward  at  the  damage  location.  It  is  uncertain  if  the 
observed  damage  would  have  resulted  in  catastrophic  blade 
failure  within  30  minutes  under  actual  flight  conditions,  or  if 
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equivalent  damage  located  at  another  spar  location  would  have 
resulted  in  the  same  outcome. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  continues  to 
address  all  DOT&E  recommendations  from  previous  testing. 
Many  of  the  recommendations,  including  the  Fire  Control 
Radar  performance  anomalies  and  the  AB3  aviator  helmet  fit 
and  visibility  display,  were  addressed  and  changed  prior  to  the 
IOT&E. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Continue  to  refine  tactics,  techniques,  and  procedures  for 
teaming  with  UASs.  Determine  the  root  cause  for  datalink 
dropouts  and  improve  the  stability  of  the  tactical  command 
datalink  for  control  of  UAS  sensors. 

2.  Consider  incorporating  improvements  to  current  threat 
warning  systems  as  they  are  developed.  Upgrade  radar  and 


laser  warning  systems  and  provide  for  adjustable  controls 
for  each  warning  system. 

3.  Address  pilot’s  confidence  concerns  with  regard  to  the 
transmission  design.  Conduct  physics  of  failure  analysis 
to  provide  an  independent  analysis  of  the  probability  of 
failure  of  the  new  tail  rotor  pinion  design.  Investigate  the 
feasibility  of  alternate  transmission  designs  that  provide 
automatic  redundant  hydraulic  and  electrical  power  in  the 
event  of  loss  of  power  to  the  tail  rotor. 

4.  Address  the  Information  Assurance  vulnerabilities 
identified. 

5.  Perform  a  structural  analysis  of  the  CMRB  to  better 
understand  the  load  carrying  capabilities  of  the  blade  that 
was  damaged  during  ballistic  testing. 


Apache 
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Armored  Tactical  Vehicles  -  Army 


Executive  Summary 

•  DOT&E  delivered  the  Family  of  Medium  Tactical  Vehicles 
(FMTV)  LFT&E  Report  to  Congress  in  October  2011. 

•  DOT&E  provided  an  Operational  Assessment  to  support 
the  re-procurement  decision  of  the  Oshkosh  FMTV.  A 
transportation  unit  can  accomplish  line  and  local  haul  convoy 
missions  using  the  Oshkosh  FMTV  in  the  same  manner  as 
the  fielded  FMTV. 

•  The  Army  is  developing  a  Modernized  Expanded  Capacity 
Vehicle  (MECV)  FTigh  Mobility  Multi-purpose  Wheeled 
Vehicle  (HMMWV)  survivability  test  series  to  determine 
available  enhanced  protection  capabilities.  In  a  parallel 
effort,  the  Marine  Corps  is  exploring  a  HMMWV 
sustainment  modification  initiative  to  restore  lost  reliability 
and  mobility  capabilities  due  to  armoring  vehicles. 

•  Emerging  Fleavy  Equipment  Transporter  (F1ET)  underbody 
test  results  indicate  that  the  FIET  Underbody  Improvement 
Kit  (UIK)  increases  crew  protection  against  under-vehicle 
threats. 

System 

FMTV 

•  The  FMTV  re-procurement  is  the  fourth  stage  of  FMTV 
progression.  These  vehicles  consist  of  the  following  light 
and  medium  variants  intended  to  operate  on-  and  off-road: 

-  The  Light  Medium  Tactical  Vehicle  (LMTV)  transports 
a  5,000-pound  payload  and  a  12,000-pound  towed 
load. 

-  The  Medium  Tactical  Vehicle  (MTV)  transports  a 
10,000-pound  payload  and  a  21,000-pound  towed  load. 

HMMWV 

•  The  HMMWV  is  a  general  purpose  tactical  wheeled 
vehicle  with  light  and  heavy  variants. 

-  The  Light  Variant  includes  the  light  utility,  weapon 
carrier,  and  ambulance  with  a  required  minimum 
payload  of  2,600  pounds. 

-  The  Heavy  Variant  includes  the  heavy  shelter 
carrier,  light  and  heavy  howitzer  towing  variant,  and 
ambulance  with  a  required  minimum  payload  of  4,550 
pounds. 

•  The  MECV  HMMWV  effort  is  intended  to  identify 
improved  underbody  crew  protection,  scalable  armor,  and 
the  ability  to  regain  automotive  performance. 

HET 

•  The  Ml 070  HET  is  an  eight- wheeled  tractor  used 
to  transport  the  Ml  main  battle  tank  and  other  large 
equipment  weighing  up  to  70  tons  to  and  from  the 
battlefield. 

•  The  HET  UIK  is  designed  to  provide  improved  underbody 
blast  protection  to  the  fielded  HETs.  The  vehicle  seats 
were  modified  in  conjunction  with  the  UIK  development. 


Mission 

FMTV 


FMTV 

(Family  of  Medium  Tactical  Vehicles) 


HMMWV 


(High  Mobility  Multi-purpose  Wheeled  Vehicle) 


HET 


(Heavy  Equipment  Transporter) 


•  The  Army  employs  the  FMTV  to  provide  multi-purpose 
transportation  and  unit  mobility  vehicles  in  maneuver, 
maneuver  support,  and  sustainment  units. 

HMMWV 


•  The  HMMWV  provides  highly  mobile  light  tactical  wheeled 
transport  for  command  and  control,  troops  and  light  cargo, 
medical  evacuation,  and  weapon  platforms  to  division 
and  below  units.  This  vehicle  is  employed  throughout 
the  battlefield  and  operates  in  off-road  and  cross-country 
environments. 


HET 


•  The  M1070  HET  is  used  to  transport,  deploy,  recover,  and 
evacuate  combat-loaded  tanks  and  other  large  tracked  and 
wheeled  vehicles. 


Major  Contractors 

FMTV  &  HET 

•  Oshkosh  Corporation  -  Oshkosh,  Wisconsin 

HMMWV 

•  AM  General  -  South  Bend,  Indiana 


ATV-Army  85 


ARMY  PROGRAMS 


Activity 

FMTV 

•  The  Army  completed  the  Production  Verification  Testing 
(PVT)  of  the  Oshkosh  FMTV  Wrecker  at  Aberdeen 
Proving  Ground,  Maryland.  The  purpose  of  the  PVT  was 
to  ensure  that  performance,  reliability,  and  maintainability 
met  the  requirements  for  the  vehicle. 

•  The  Army  Test  and  Evaluation  Command  completed 
the  FMTV  Developmental/Operational  Test  (DT/OT)  at 
Aberdeen  Proving  Ground  in  June  2011  in  accordance 
with  the  DOT&E -approved  test  plan. 

•  DOT&E  delivered  the  FMTV  LFT&E  Report  to  Congress 
in  October  2011. 

•  DOT&E  provided  an  Operational  Assessment  in 
October  2011  to  support  the  re-procurement  decision  of 
the  FMTV  by  Oshkosh. 

HMMWV 

•  In  February  2012,  the  Army  Test  and  Evaluation 
Command  (ATEC)  completed  developmental  testing  and 
a  user  evaluation  of  the  M997A3  HMMWV  Ambulance 
at  Aberdeen  Proving  Ground,  Maryland,  to  support  the 
procurement  of  500  vehicles  for  the  Army  National 
Guard. 

•  The  Army  is  developing  an  MECV  HMMWV 
survivability  test  series  to  determine  available  enhanced 
protection  capabilities.  In  a  parallel  effort,  the  Marine 
Corps  is  exploring  a  HMMWV  sustaimnent  modification 
initiative  to  restore  lost  reliability  and  mobility 
capabilities  due  to  armoring  vehicles. 

HET 

•  ATEC  conducted  four  system-level  underbody  blast 
tests  against  the  FIET  U1K  at  the  Aberdeen  Test  Center, 
Aberdeen,  Maryland.  The  results  compare  the  blast 
protection  of  the  base  and  up-armored  FIET. 

•  The  M1070  FIET  Live  Fire  test  series  included  threats 
above  Mine  Resistant  Ambush  Protected  All-Terrain 
Vehicle  levels. 

•  The  program  procured  55  FIET  UIKs  in  August  2011. 


Assessment 

FMTV 

•  Based  on  DT/OT  results,  a  transportation  unit  can 
accomplish  line  and  local  haul  convoy  missions  using  the 
Oshkosh  FMTV  in  the  same  manner  as  the  fielded  FMTV. 

•  During  PVT,  the  Oshkosh  FMTV  Wrecker  demonstrated 
8,000  Mean  Miles  Between  Flardware  Mission  Failure 
(MMBF1MF)  versus  its  operational  requirement  of 
5,000  MMBF1MF.  The  Wrecker  is  capable  of  recovering 
and  towing  wheeled  vehicles  such  as  the  HMMWV,  5-ton 
truck  series,  and  FMTV  vehicles  over  a  variety  of  terrain 
and  surfaces. 

HMMWV 

•  Contractor  results  from  the  ballistic  tests  on  the  potential 
MECV  designs  demonstrated  that  improvements  in 
survivability  are  feasible.  The  Army  will  retest  these 
designs  during  the  MECV  HMMWV  survivability  test 
series. 

•  The  M997A3  HMMWV  Ambulance  contributes  to  the 
accomplishment  of  the  medical  ground  evacuation  mission 
in  support  of  Homeland  Defense  and  Homeland  Security 
operations  based  on  the  results  of  developmental  testing  and 
user  evaluation.  The  medical  crews  effectively  collected, 
provided  en-route  treatment,  and  transferred  patients  from 
simulated  points  of  injury  to  a  treatment  facility  utilizing 
the  Ambulance.  The  M997A3  HMMWV  Ambulance 
demonstrated  a  Mean  Miles  Between  Operational  Mission 
Failure  of  3,053  miles  during  DT/OT  versus  the  1,637  miles 
requirement. 

HET 

•  Emerging  underbody  test  results  indicate  that  the  M1070 
HET  U1K  increases  crew  protection  against  under-vehicle 
strikes. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed  all 

previous  recommendations. 

•  FY12  Recommendations.  None. 
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Bradley  Engineering  Change  Proposal  (ECP) 


Executive  Summary 

•  The  Bradley  Engineering  Change  Proposal  (ECP)  LFT&E  was 
initiated  to  evaluate  the  vulnerability  of  ECP  upgrades,  which 
intend  to  restore  ground  clearance  and  integrate  new  network 
technologies  and  to  evaluate  the  vulnerability  of  urgently 
fielded  survivability  kits,  which  are  now  part  of  the  M2A3 
configuration. 

•  In  September  2012,  the  Army  conducted  two  underbody 
blast  tests  on  the  M2A3  Infantry  Fighting  Vehicle  with 
ECP1  components  that  revealed  severe  vehicle  and  occupant 
vulnerabilities.  These  two  tests  are  not  sufficient  to  address  all 
of  the  critical  survivability  concerns  with  the  Bradley  Fighting 
Vehicle  Systems  (BFVS).  Additional  testing  is  required. 

System 

•  The  Bradley  ECP1  intends  to  restore  ground  clearance  with 
upgrades  to  the  suspension  and  track  and  ECP2  intends  to 
integrate  network  technologies  as  they  become  available  for 
the  following  three  variants  of  the  BFVS: 

-  M2A3  Infantry  Fighting  Vehicle 

-  M3  A3  Cavalry  Fighting  Vehicle 

-  Bradley  Fire  Support  Team  with  Fire  Support  Sensor 
System 

•  The  Bradley  Urban  Survivability  Kit  I,  II,  and  III  and 
add-on-armor  kit  intend  to  improve  vehicle  and  crew 
survivability.  These  kits  were  urgently  fielded  for  Operation 
Iraqi  Freedom  and  are  now  part  of  the  M2  A3  configuration. 


Mission 

Combatant  Commanders  employ  BFVS-equipped  Armor  Brigade 
Combat  Teams  to  provide  protected  transport  of  Soldiers;  provide 
overwatching  fires  to  support  dismounted  infantry  and  suppress 
an  enemy;  and  perform  missions  to  disrupt  or  destroy  enemy 
military  forces  and  control  land  areas. 

Major  Contractor 

BAE  Systems  Land  and  Armaments  -  Sterling  Heights,  Michigan 


Activity 

•  The  Army  initiated  the  Bradley  Engineering  Change  Proposal 
(ECP)  LFT&E  to  evaluate  the  vulnerability  of  ECP  upgrades, 
which  include  an  upgraded  suspension,  upgraded  track,  and 
new  network  technologies  and  to  evaluate  the  vulnerability 
of  urgently  fielded  survivability  kits,  now  part  of  the  M2A3 
configuration. 

•  DOT&E  approved  the  Phase  1  Bradley  ECP  LFT&E  strategy 
in  August  2012. 

•  In  September  2012,  the  Army  conducted  two  underbody 
blast  tests  at  the  Aberdeen  Test  Center  on  the  M2A3  Infantry 
Fighting  Vehicle  with  ECP1  components  to  characterize  the 
system’s  vulnerability. 

•  The  Program  Office  wrote  a  draft  Test  and  Evaluation  Master 
Plan  (TEMP)  that  outlined  the  Bradley  ECP1  operational  and 
live  fire  test  plans,  test  methods,  resources,  and  evaluation 
plans.  The  draft  TEMP  is  in  staffing  for  review  and  approval. 
The  Program  Office  will  update  the  TEMP  in  1QFY15  for 
ECP2. 


Assessment 

•  Results  from  the  first  two  underbody  blast  tests  with  realistic 
threats  (as  opposed  to  the  outdated  underbody  requirement) 
conducted  in  September  2012  revealed  severe  vehicle  and 
occupant  vulnerabilities,  which  the  Army  plans  to  examine 
for  possible  improvement.  The  first  two  underbody  tests  are 
not  sufficient  to  address  all  of  the  critical  BFVS  survivability 
concerns. 

•  Results  from  the  first  two  underbody  blast  tests  also 
demonstrate  that  survivability  modifications  to  the  Bradley 
Fighting  Vehicle  would  be  required  if  it  is  chosen  as  the 
platform  for  the  Armored  Multi-purpose  Vehicle  (AMPV)  to 
meet  the  draft  AMPV  survivability  requirement. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 
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•  FY12  Recommendations. 

1 .  The  Army  must  provide  a  comprehensive  live  fire  strategy 
for  DOT&E  approval,  which  includes  plans  for  additional 
tests  required  to  comprehensively  evaluate  the  force 
protection  and  survivability  provided  by  the  Bradley  Urban 
Survivability  Kit  I,  II,  and  III;  add-on-armor  kit;  ECP1;  and 
ECP2  components. 

2.  The  Army  should  correct  the  vulnerabilities  identified 
during  underbody  blast  testing  prior  to  using  the  vehicles  in 
combat  situations  in  which  underbody  threats  are  prevalent. 
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Common  Remotely  Operated  Weapon  Station  (CROWS) 


Executive  Summary 

•  The  Army  conducted  an  Initial  Operational  Test  (IOT)  in 
2009. 

•  Early  in  FY12,  the  Army  Acquisition  Executive  notified  the 
USD(AT&L)  that  the  CROWS  program  was  expected  to 
become  a  Major  Defense  Acquisition  Program  with  the  Army 
as  the  lead  Service.  This  designation  caused  the  program  to 
come  under  DOT &E  oversight. 

•  In  June  2012,  Program  Executive  Office  (PEO)  Soldier 
requested  that  DOT&E  prepare  an  OT&E  report  to  Congress 
to  support  a  September  2012  production  decision  for  the 
procurement  of  the  final  1,212  CROWS  systems. 

•  In  September  2012,  DOT&E  provided  Congress  with  an 
assessment  of  the  operational  effectiveness  and  operational 
suitability  of  the  CROWS  mounted  on  Up- Armored  High 
Mobility  Multi-purpose  Wheeled  Vehicles  (UAHs)  based  upon 
the  results  of  the  IOT,  developmental  testing,  and  CROWS 
New  Equipment  Training  (NET)  at  Fort  Campbell,  Kentucky. 

•  The  CROWS  system  is  operationally  effective.  The  CROWS 
target  acquisition  and  engagement  capabilities  enable 

units  to  detect  and  engage  targets  at  long  range  while  both 
on-the-move  and  stationary  relative  to  non-CROWS  equipped 
units.  CROWS  operators  are  provided  protection  over 
Objective  Gunner  Protection  Kit  (OGPK)  gunners  because  of 
the  CROWS  ability  to  fire  remotely.  Nonetheless,  CROWS 
has  some  limitations  in  comparison  with  the  OGPK  due  in 
part  to  the  limited  fields  of  view  of  the  CROWS  daytime  and 
nighttime  sensors. 

•  The  CROWS  system  is  operationally  suitable.  During 
IOT,  the  CROWS  exceeded  its  reliability  requirement.  The 
CROWS-equipped  UAH  demonstrated  the  capability  to 
perform  its  mission  essential  functions  of  move,  shoot,  and 
communicate. 

System 

•  CROWS  is  a  gunner-operated  system  that  provides  the 
capability  to  remotely  aim  and  fire  the  MK19  Grenade 


Ml 53,  CROWS 
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Machine  Gun  (GMG),  M2  Machine  Gun,  M240  Machine  Gun, 
or  the  M249  Machine  Gun  from  a  stationary  platform  or  while 
on-the-move. 

•  The  M153  CROWS  consists  of  weapons  cradles,  traverse  and 
elevation  drives,  weapon  interface,  weapon  remote  charger, 
ammunition  container  and  feed  system,  laser  range  finder, 
day/night  viewing  and  sighting  unit,  joystick,  and  remote  fire 
control  and  display  unit. 

Mission 

Gunners  within  a  vehicle  crew  or  in  a  stationary  battle  position 
use  CROWS  to  improve  their  weapon’s  performance  through 
enhanced  target  acquisition,  identification,  and  engagement 
capabilities  while  firing  remotely.  Units  equipped  with  CROWS 
include  Infantry,  Artillery,  Armor,  Cavalry,  Engineer,  Chemical, 
and  Military  Police. 

Major  Contractor 

Kongsberg  Defense  Corporation  -  Johnstown,  Pennsylvania 


Activity 

•  The  Army  conducted  an  IOT  in  2009.  At  the  time  the  IOT  was 
conducted,  CROWS  was  an  Acquisition  Category  (AC AT)  II 
program  and  was  not  on  DOT&E  oversight.  The  Army 

Test  and  Evaluation  Command  (ATEC)  was  responsible  for 
approving  the  test  plan,  conducting  the  IOT,  and  reporting  its 
evaluation  to  the  Army. 

•  ATEC  conducted  the  IOT  at  Fort  Carson,  Colorado,  from 
October  -  November  2009.  For  the  IOT,  the  performance  of 
a  CROWS-equipped  Military  Police  (MP)  unit  was  compared 


to  an  MP  platoon  equipped  with  OGPKs.  All  four  types  of 
weapons  were  used  during  the  test.  Representatives  from 
DOT&E  visited  the  site  during  the  conduct  of  the  IOT  in  2009 
to  monitor  test  execution  adequacy. 

•  Early  in  FY12,  the  Army  Acquisition  Executive  notified 
USD(AT&L)  that  the  CROWS  program  was  expected  to 
reach  an  ACAT  1  funding  level  for  the  procurement  year. 

In  March  2012,  the  USD(AT&L)  designated  the  CROWS 
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program  as  an  ACAT  IC  Major  Defense  Acquisition  Program 
with  the  Army  as  the  lead  Service. 

•  In  June  2012,  PEO  Soldier  requested  that  DOT&E  prepare 
an  OT&E  report  to  Congress  to  support  a  September  2012 
production  decision  for  the  procurement  of  the  final 
1,212  CROWS  systems. 

•  In  August  2012,  DOT&E  observed  the  revised  CROWS 
NET  at  Fort  Campbell,  Kentucky.  Thirty-two  Soldiers  from 
different  units  at  Fort  Campbell  participated. 

•  In  September  2012,  DOT&E  provided  Congress  with  an 
assessment  of  the  operational  effectiveness  and  operational 
suitability  of  the  CROWS  mounted  on  UAHs  based  upon 
the  results  of  the  IOT,  developmental  testing,  and  the  revised 
CROWS  NET. 

Assessment 

•  Representatives  from  DOT&E  monitored  the  Army’s  IOT 
at  Fort  Carson,  Colorado,  in  2009  and  assessed  that  it  was 
adequately  executed. 

•  The  CROWS  system  is  operationally  effective.  The  CROWS 
target  acquisition  and  engagement  capabilities  enable 

units  to  detect  and  engage  targets  at  long  range  while  both 
on-the-move  and  stationary  relative  to  non-CROWS  equipped 
units. 

-  The  CROWS  provides  enhanced  lethality  and  is  more 
accurate  while  firing  on-the-move  at  long  ranges  than  a 
crew-served  weapon  fired  by  a  gunner  using  the  OGPK. 

-  CROWS  operators  are  provided  protection  over  OGPK 
gunners  because  of  the  ability  to  remotely  fire  CROWS. 

A  unit  with  CROWS-equipped  vehicles  can  synchronize 
target  acquisition,  maneuver,  and  provide  responsive 
fires  during  missions  such  as  Convoy  Security,  Route 
Reconnaissance,  and  Overwatch. 

•  The  shortcomings  of  CROWS  identified  during  IOT  were  that 
the  operator  and  crew  struggled  to  detect  enemy  personnel 
close  to  the  vehicle  and  to  maintain  situational  awareness  of 
their  surroundings.  Additionally,  they  were  slower  than  the 
OGPK-equipped  UAHs  in  determining  the  location  of  enemy 
fire  as  the  CROWS  operator  and  crew  lacked  the  visual  and 
auditory  cues  necessary  to  stay  in  the  firefight. 


•  The  CROWS  daylight  sight  provides  a  47-degree  field  of  view 
and  its  minimum  focus  distance  is  2  meters.  The  CROWS 
thermal  sighting  only  provides  a  narrow  10-degree  field 

of  view.  These  capabilities  limit  the  gunner/operator  from 
rapidly  acquiring  dispersed  targets,  whereas  a  gunner  operating 
the  OGPK  can  rapidly  scan  for  and  detect  close-in  and  widely 
dispersed  targets. 

•  The  CROWS  system  is  operationally  suitable.  During 
IOT,  the  CROWS  exceeded  its  reliability  requirement.  The 
CROWS-equipped  UAH  demonstrated  the  capability  to 
perform  its  mission  essential  functions  of  move,  shoot,  and 
communicate. 

•  The  revised  2012  NET  is  improved  over  that  conducted  in 
2009.  The  new  program  of  instruction  (POI)  incorporates 
expanded  hands-on,  situational  awareness,  safety,  and  gunnery 
live-fire  exercises.  The  POI  now  includes  lessons  on  the 
correct  method  to  establish  no-fire  and  no-traverse  zone. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendations.  The  CROWS  program  manager 
should: 

1 .  Conduct  follow-on  operational  testing  to  evaluate  the 
effectiveness  and  suitability  of  CROWS  as  it  is  integrated 
for  use  on  combat  vehicles  in  addition  to  the  High  Mobility 
Multi-purpose  Wheeled  Vehicle  and  Mine  Resistant 
Ambush  Protected  vehicle. 

2.  Investigate  increasing  the  field  of  view  of  the  CROWS 
daytime  and  thermal  sights  to  improve  CROW  operator 
determination  of  enemy  location.  The  CROWS  imaging 
sights  have  limited  field  of  view,  which  affects  the  crew’s 
ability  to  acquire  and  engage  the  enemy. 

3.  Test  to  confirm  the  updated  fire  tables  corrective  action 
improve  the  MK19  accuracy  with  CROWS  in  a  desert 
environment. 

4.  Validate  that  link  guide  corrective  action  deflects  expended 
cartridge  cases  and  links. 
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Distributed  Common  Ground  System  -  Army  (DCGS-A) 


Executive  Summary 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  conducted 
a  Developmental  Test/Early  User  Test  (DT/EUT)  on  the 
Distributed  Common  Ground  System  -  Army  (DCGS-A) 
Software  Baseline  1.0  (DSB  1.0)  from  August  through 
September  2011,  to  provide  information  for  the  Milestone  C 
decision  in  February  2012. 

•  DOT&E  published  a  DCGS-A  DSB  1.0  Operational 
Assessment  Report  in  January  2012.  ATEC  conducted  the  test 
in  a  non-operationally  representative  laboratory  environment. 
Based  on  the  DT/EUT  data,  DOT&E  evaluated  DSB  1.0 

to  be  sufficiently  mature  to  enter  production  in  preparation 
for  IOT&E,  but  recommended  improvements  to  Tactics, 
Techniques,  and  Procedures  (TTPs);  individual  and  collective 
training;  and  system  reliability. 

•  ATEC  conducted  the  DSB  1.0  IOT&E  from  May  through 
June  2012,  utilizing  an  operationally  representative  field 
configuration. 

•  DOT&E  published  a  DCGS-A  DSB  1 .0  IOT&E  Report  in 
October  2012  that  evaluated  DSB  1.0  to  be  not  operationally 
effective,  not  operationally  suitable,  and  not  survivable  based 
on  the  IOT&E  data. 

-  During  IOT&E,  the  physical  configuration  of  DSB  1.0 
forced  unnecessary  foot  traffic  and  data  exchange  through 
the  security  cross-domain  guard  since  most  of  the  data 
needed  for  the  fusion  is  Secret,  but  the  fusion  capability 
was  only  in  the  Sensitive  Compartmented  Information 
(SCI)  enclave.  Physical  barriers  and  security  procedures 
between  the  SCI  enclave  and  the  remainder  of  the  DSB  1.0 
system  inhibited  exchange  and  fusion  of  data  to  the  point 
where  DOT&E  assessed  the  DSB  1.0  system  to  be  not 
operationally  effective. 

-  Additionally,  software  faults  within  the  SCI  enclave  were  a 
primary  factor  in  the  evaluation  of  the  DSB  1.0  system  as 
not  operationally  suitable. 

•  The  Army  reconfigured  DSB  1.0  without  the  SCI  enclave  to 
mitigate  the  effectiveness  and  suitability  shortfalls  identified 
in  the  IOT&E  report  and  demonstrated  fixes  to  the  critical 
Information  Assurance  (IA)  shortfalls  that  led  to  the  evaluation 
that  the  system  was  not  survivable.  The  reconfigured  package 
is  called  Release  1. 

•  DOT&E  released  a  memorandum  in  November  2012  that 
stated  Release  1  will  provide  users  with  capabilities  at  least  as 
good  as  those  provided  by  the  current  systems. 

System 

•  DSB  1.0  establishes  the  architecture  that  will  provide 
an  organic  net-centric  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  capability.  DSB  1.0  integrates  three 
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major  existing  systems  and  provides  new  SCI  components  to 
brigade  combat  teams.  The  major  components  include  the 
following: 

-  V3.1.6  ISR  Fusion  Server  is  the  same  as  the  currently 
fielded  DCGS-A  v3. 1.6  and  organizes  and  processes  raw 
intelligence  reports  into  coherent  information  at  the  Secret 
classification  level. 

-  DCGS-A  Enabled  Common  Ground  Station  provides 
multisensory  imagery  intelligence  data  processing  and 
evaluation  capability. 

-  DCGS-A  Enabled  Digital  Topographic  Support 
System  -  Light  provides  geospatial  analysis  and 
production  capability. 

-  Mobile  Basic  Enclave  contains  new  components  designed 
to  provide  signal  intelligence  capability  and  intelligence 
fusion  and  analysis  capability  for  Top  Secret  information 
and  SCI. 

•  DCGS-A  allows  users  to  collect,  process,  fuse,  and  display 
intelligence  on  six  types  of  enemy  entities:  individuals,  units, 
equipment,  facilities,  events,  and  organizations. 

•  DCGS-A  is  the  information-  and  intelligence-processing 
centerpiece  of  the  Army  ISR  framework  and  is  the  enabler 
for  all  intelligence  functions  at  the  Division,  Brigade  Combat 
Team,  Maneuver  Battalion,  and  Company  levels. 

Mission 

Army  intelligence  analysts  use  DSB  1.0  to  support  six  Mission 

Command  Capabilities: 

•  Display  and  share  relevant  information 

•  Provide  a  standard  and  shareable  geospatial  foundation 

•  Collaborate  in  voice,  text,  data,  and  video  modes 

•  Execute  running  estimates  of  enemy  force  progress 

•  Provide  mission  rehearsal  and  training  support 

•  Interoperate  across  the  joint,  interagency,  intergovernmental, 
and  multinational  forces 

Major  Contractor 

Northrop  Grumman  Electronic  Systems  -  Linthicum,  Maryland 
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Activity 

•  From  August  through  September  2011,  ATEC  conducted 
the  DT/EUT  utilizing  a  non-operationally  representative 
system  configuration  at  the  Intelligence  Systems  Integration 
Laboratory  at  Fort  Huachuca,  Arizona.  In  the  laboratory 
configuration,  the  SCI  enclave  and  the  rest  of  the  DSB  1.0 
system  were  connected  by  a  single  secured  door.  Testing  was 
conducted  in  accordance  with  a  DOT&E -approved  test  plan. 

•  DOT&E  published  the  DCGS-A  DSB  1.0  Operational 
Assessment  Report  in  January  2012  informing  the  Milestone 
Decision  Authority  on  the  test  results  of  the  DT/EUT. 

•  The  Milestone  Decision  Authority  signed  the  Acquisition 
Decision  Memorandum  in  March  2012,  approving  limited 
deployment  for  IOT&E. 

•  From  May  through  June  2012,  ATEC  conducted  the  IOT&E 
at  Fort  Stewart,  Georgia,  with  the  4th  Brigade  Combat  Team, 
3rd  Infantry  Division  operating  the  system  in  an  operationally 
representative  field  configuration.  In  the  fielded  configuration, 
the  SCI  enclave  and  the  rest  of  the  DSB  1.0  system  were 
separated  physically  and  by  security  barriers  (concertina  wires 
and  guarded  entrances).  Testing  was  conducted  in  accordance 
with  a  DOT&E-approved  test  plan. 

•  In  October  2012,  DOT&E  provided  an  IOT&E  report  to  the 
Milestone  Decision  Authority  and  Congress. 

•  The  Army  reconfigured  the  DSB  1.0  without  the  SCI  enclave 
to  mitigate  the  effectiveness  and  suitability  shortfalls  identified 
in  the  IOT&E  report,  and  demonstrated  fixes  to  the  critical  I A 
shortfalls.  The  reconfigured  package  is  called  Release  1 . 

•  DOT&E  released  a  memorandum  in  November  2012  that 
stated  Release  1  will  provide  users  with  capabilities  at  least  as 
good  as  those  provided  by  the  current  systems. 

Assessment 

•  The  DT/EUT  demonstrated  that  the  DSB  1.0  system  had 
shortfalls  but  was  sufficiently  mature  to  enter  production  in 
preparation  for  IOT&E. 

-  During  the  DT/EUT,  test  data  showed  operators  using 
DSB  1.0  could  execute  all  key  missions,  although  software 
limitations  forced  the  users  to  manually  fuse  data  on  two 
of  the  six  entity  types.  DSB  1.0  could  perform  fusion  for 
unit,  equipment,  facility,  and  event  entities,  but  not  for 
individuals  and  organizations.  The  software  modules  to 
fuse  data  on  each  entity  type  are  unique  due  to  the  different 
data  sets  associated  with  each  entity. 

-  DSB  1.0  was  not  reliable  due  to  software  problems. 

The  Mean  Time  Between  Essential  Capability  Failure 
(MTBECF)  of  3.8  hours  fell  short  of  the  160-hour 
requirement.  The  primary  cause  for  all  reliability  failures 
was  software  (44  of  49  for  all  failures,  15  of  16  for 
essential  capability  failures). 

-  The  TTPs  and  training  were  not  mature  and  needed 
improvement  prior  to  IOT&E.  The  TTP  and  training 
shortfalls  during  DT/EUT  precluded  comprehensive 
operational  evaluation  of  the  end-to-end  mission 
sequence.  These  shortfalls  channelized  testers’  attention 
and  precluded  identification  of  the  non-trivial  problems 


(discovered  later  during  IOT&E)  that  were  associated  with 
the  physical  separation  of  the  SCI  enclave  and  the  rest  of 
the  DSB  1.0  system. 

-  The  Army  did  not  evaluate  DSB  1.0  survivability  against 
cyber  threats  during  DT/EUT. 

•  The  change  from  a  laboratory  configuration  to  the  fully  fielded 
configuration  in  IOT&E  significantly  altered  the  test  results 
from  those  seen  during  DT/EUT.  The  IOT&E  results  showed 
DSB  1.0  is  not  operationally  effective,  not  operationally 
suitable,  and  not  survivable. 

-  The  system  is  not  effective  because  it  inhibits  effective 
workflows  for  the  development  of  intelligence  products  to 
support  operations.  The  system  configuration,  as  tested, 
placed  the  intelligence  fusion  capability  in  the  SCI  (high) 
enclave  even  though  most  of  the  data  to  be  fused  are 
contained  in  the  Secret  (low)  side.  Additionally,  collection 
management  tools  reside  in  the  high  side,  but  collection 
managers  need  to  work  closely  with  the  brigade  operations 
staff  on  the  low  side.  Human  intelligence  tools  are  split 
between  the  high  side  and  low  side,  but  human  intelligence 
analysts  manage  and  interview  their  sources  on  the  low 
side.  The  need  for  constant  collaboration  between  high  and 
low  side  personnel  and  the  utilization  of  fusion  capabilities 
contained  only  on  the  high  side,  in  addition  to  physical 
separation  and  security  procedures,  inhibited  access  to 
capabilities  and  ultimately  severely  degraded  the  ability  to 
generate  a  single,  fused  intelligence  plot. 

-  The  system  is  not  suitable  because  operators  performing 
key  mission  tasks  are  interrupted  with  server  reboots 

or  reliability  failures,  on  average,  every  8  hours,  and 
operators  frequently  need  to  recreate  lost  work  as  a 
result  of  computer  resets.  Training  for  the  targeting  and 
signal  intelligence  analysts  using  the  new  capabilities  is 
inadequate. 

-  The  system  is  not  survivable  against  cyber  threats  and  does 
not  provide  adequate  protection  and  detection  against  them. 

•  As  reconfigured,  Release  1  provides  useful  capability  for  the 
Army  users,  but  does  not  fully  satisfy  the  Key  Performance 
Parameters  (KPPs)  for  DSB  1.0. 

-  Release  1  does  not  fully  meet  the  Net-Ready  KPP  since  the 
information  exchange  requirement  that  involves  the  SCI 
component  is  not  satisfied. 

-  Release  1  does  not  fully  meet  the  Fusion  KPP  since  the 
software  that  performs  semi-automated  fusion  was  hosted 
on  the  SCI  enclave.  However,  Army  users  can  perform 
fusion  with  manual  assist. 

-  Release  1  critical  IA  shortfalls  have  been  corrected. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendation. 

1 .  The  Army  should  conduct  operational  testing  of  all  releases 
of  DCGS-A  Increment  1  to  be  deployed  for  operational  use. 
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Excalibur  XM92  Precision  Engagement  Projectiles 


Executive  Summary 

•  As  of  July  2012,  the  Army  has  fired  601  Excalibur  la 
projectiles  in  operational  theaters  with  the  Program  Office 
reporting  90  percent  reliability. 

•  In  February  2012,  technical  challenges  with  the  Increment  lb 
projectiles  caused  reliability  concerns  with  the  base  assembly. 
This  resulted  in  the  program  changing  the  configuration  of  the 
Increment  lb  projectiles. 

•  Excalibur  Increment  lb  is  scheduled  for  a  December  2012 
Milestone  C  decision. 

System 

•  Excalibur  is  a  family  of  precision-guided,  extended-range, 
155-millimeter  artillery  projectiles.  The  projectiles  are 
fin-stabilized  and  glide  to  their  target. 

•  The  Army  plans  to  develop  three  Excalibur  variants: 

-  High-Explosive,  Unitary  (Block  I) 

-  Smart  (Block  II) 

-  Discriminating  (Block  III) 

•  All  variants  use  GPS  and  an  Inertial  Measurement  Unit  (IMU) 
to  attack  point  targets  with  accuracy  of  less  than  10  meters 
from  the  desired  aim  point  (in  an  unjammed  environment). 

•  The  Army  is  developing  the  High-Explosive,  Unitary  (Block  I) 
projectile  in  three  spirals  of  increasing  capability  (la- 1,  la-2, 
and  lb).  The  la-1  projectiles  use  aerodynamic  lift  generated 
by  canards  to  extend  range  out  to  24  kilometers  without  the 
maximum  propellant  charge.  The  la-2  and  lb  projectiles  add 
base  bleed  technology  and  use  of  the  maximum  propellant 
charge  to  further  increase  range  to  beyond  35  kilometers.  The 
Army  intends  for  the  Increment  lb  projectiles  to  improve 
reliability  and  reduce  cost. 


Mission 

•  Field  Artillery  units  use  Excalibur  to  attack  enemy  targets  in 
support  of  maneuver  operations  at  a  greater  range  and  with 
increased  accuracy  than  standard  high-explosive  munitions. 

•  Field  Artillery  units  use  Excalibur  to  support  the  close  fight  in 
urban  and  complex  environments,  striking  critical  targets  that 
must  be  engaged  at  extended  ranges  or  in  areas  where  minimal 
collateral  damage  is  desired. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

Increment  la 

•  Units  in  Operation  Enduring  Freedom  (OEF)  have  been 
using  Excalibur  Increments  la-1  and  la-2  since  February 
2008.  As  of  July  2012,  Field  Artillery  units  have  fired  601 
Excalibur  projectiles  in  either  Operation  Iraqi  Freedom  or 
OEF. 

Increment  lb 

•  In  February  2012,  technical  challenges  with  the 
Increment  lb  projectiles  caused  reliability  concerns  with  the 
base  assembly.  This  resulted  in  the  program  changing  the 
configuration  of  the  Increment  lb  projectiles,  reverting  to 
the  la  spinning  base  assembly  and  warhead  design. 

•  In  June  2012,  the  contractor  fired  six  Increment  lb 
projectiles  in  the  contractor  Gun  Hardening  P2  (GH  P2) 
test  at  Yuma  Test  Center  (YTC),  Arizona,  for  reliability  and 
accuracy  testing.  All  GH  P2  test  projectiles  reached  their 


targets  with  2.9-meter  median  radial  miss  distance.  All  six 
projectiles  reached  their  intended  targets  and  detonated. 

•  In  September  2012,  YTC  personnel  fired  12  Increment 
lb  projectiles  in  developmental  testing  for  Sequential 
Environmental  Test-Safety  1  (SET-SI).  All  rounds 
completed  environmental  conditioning  (logistic  and  tactical 
vibration  and  thermal  soak)  prior  to  firing,  and  were  fired 
at  pressures  above  the  normal  operational  envelope.  Two 
projectiles  failed  to  guide  and  traveled  to  the  ballistic 
impact  point.  Ten  projectiles  reached  their  targets  with  a 
1.5-meter  median  radial  miss  distance. 

•  In  October  2012,  the  Army  fired  25 
environmentally-conditioned  projectiles  at  YTC  to  support 
validation  of  the  ballistic  firing  tables,  verify  that  Excalibur 
Increment  lb  meets  performance  requirements,  and  provide 
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live  fire  target  assessment  data  (Sequential  Environmental 
Test-Performance  1  (SET-PI)). 

•  The  Increment  lb  Milestone  C  decision  is  scheduled 
for  December  2012,  and  the  IOT&E  is  scheduled  for 
April  2013. 

Assessment 

Increment  la 

•  Excalibur  Increment  la-2  reliability  in  OEF  is  90  percent 
(according  to  the  Program  Office)  and  Lot  Acceptance 
Tests  conducted  in  November  and  December  2011,  and 
February  2012,  had  24  of  24  reliable  projectiles. 

Increment  lb 

•  Increment  lb  has  the  same  base  and  warhead  as 
Increment  la-2,  and  is  expected  to  have  the  same  terminal 
ballistic  characteristics  and  lethality  as  Increment  la-2. 

•  Since  the  GH  P2  median  radial  miss  distance  is  comparable 
to  the  Increment  la  IOT&E  (unjammed)  miss  distance  (3 
meters).  Increment  lb  is  anticipated  to  meet  the  10-meter 
accuracy  requirement  and  be  as  accurate  as  Increment  la. 

•  Since  external  differences  between  Excalibur  Increments 
la  and  lb  are  minor  and  units  are  expected  to  use  the 


same  tactics,  techniques,  and  procedures,  Increment  lb  is 
anticipated  to  be  as  effective  as  Increment  la. 

•  Of  the  25  SET-PI  rounds  fired,  22  hit  the  target  with  a 
median  miss  distance  of  1.38  meters  and  3  were  duds.  With 
regards  to  reliability,  there  were  six  failures  in  31  rounds 
(including  the  GH-2  rounds).  This  gives  a  point  estimate 
of  0.81  and  a  21.8  percent  confidence  that  they  have  met 
their  growth  curve  target  of  0.84.  The  program  will  need 
to  complete  root  cause  analysis  before  an  assessment  can 
be  made  as  to  whether  the  program  can  meet  its  90  percent 
reliability  requirement. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  is  addressing 

the  previous  recommendations. 

•  FY12  Recommendations. 

1.  The  Army  should  continue  Increment  lb  reliability  growth 
through  completion  of  the  developmental  test. 

2.  The  program  will  need  to  complete  root  cause  analysis 
before  an  assessment  can  be  made  as  to  whether  the 
program  can  meet  its  90  percent  reliability  requirement. 
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Global  Combat  Support  System  -  Army  (GCSS-Army) 


Executive  Summary 

•  The  Global  Combat  Support  System  -  Army  (GCSS-Army) 
Release  1 . 1  is  operationally  effective,  operationally 
suitable,  and  survivable  against  cyber  threats.  DOT&E 
recommendations  from  the  June  2012  IOT&E  report  included 
continued  test  and  evaluation  for  future  upgrades,  monitoring 
scalability  as  the  user  population  increases,  and  achieving 
financial  auditability  no  later  than  2017. 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  will 
participate  in  the  Lead  Site  Verification  Test  in  January  2013  to 
evaluate  GCSS-Army’s  operational  effectiveness,  operational 
suitability,  and  survivability  for  use  by  the  Army  National 
Guard,  Army  Reserves,  and  Directorate  of  Logistics. 

System 

•  GCSS-Army  is  an  information  technology  system  made  up  of 
commercial  off-the-shelf  and  non-developmental  software  and 
server  hardware. 

•  The  core  functionality  of  GCSS-Army  comes  from  the 
adaptation  of  a  commercially-available  Enterprise  Resource 
Planning  (ERP)  system.  The  ERP  system  integrates  internal 
and  external  management  information  across  an  entire 
organization,  including  finance/accounting,  manufacturing, 
sales  and  service,  and  customer  relationship  management,  and 
automates  this  activity  with  an  integrated  software  application. 

•  The  hardware  component  of  GCSS-Army  is  limited  to  the 
production  server  in  Redstone,  Alabama,  and  the  Continuity  of 
Operation  server  in  Radford,  Virginia. 

•  The  GCSS-Army  program  includes  the  Army  Enterprise 
Systems  Integration  Program  (AESIP)  that  provides  the 
enterprise  hub  services,  centralized  master  data  management, 
and  cross  functional  business  intelligence  and  analytics  for  the 
Army  ERP  solutions,  including  the  General  Fund  Enterprise 
Business  System  and  Logistics  Modernization  Program. 

•  GCSS-Army  executes  finance  actions  and  thus  is  subject  to  the 
2010  National  Defense  Authorization  Act  requirements  to  be 
auditable  by  2017. 


Providing  Soldiers  a 
single  system  to 
accomplish  a  wide 
range  of  logistics 
missions 


Mission 

Army  logisticians  will  use  this  system  to  access  information 
and  exchange  operational  logistics  data  related  to  tactical 
maintenance,  materiel  management,  property  accountability, 
tactical  financial  management,  and  logistics  planning. 

Major  Contractors 

•  ERP  Solution  Component:  Northrop  Grumman  Space  and 
Mission  Systems  Corporation  -  Carson,  California 

•  AESIP  Component:  Computer  Sciences  Corporation  -  Falls 
Church,  Virginia 


Activity 

•  DOT&E  published  an  IOT&E  report  on  GCSS-Army 
Release  1.1  in  June  2012  based  on  the  following  three  test 
events.  These  events  were  conducted  in  accordance  with  the 
DOT&E-approved  test  plan. 

-  ATEC  Release  1.1  testing  conducted  from  August  through 
October  2011.  The  2nd  Heavy  Brigade  Combat  Team, 

1st  Armored  Division,  at  Fort  Bliss,  Texas,  and  Defense 
Finance  and  Accounting  Office  at  Rome,  New  York, 
participated  in  the  IOT&E. 


-  The  GCSS-Army  Cyber  Threat  Test  conducted  by  the 
U.S.  Army  Threat  System  Management  Office  from 
September  2011,  and  the  Program  Management  Office’s 
corrective  actions  up  through  March  2012. 

-  A  Continuity  of  Operation  demonstration  conducted 
by  the  GCSS-Army  and  AESIP  product  manager  in 
November  2011,  which  evaluated  the  system’s  ability  to 
restore  operations  in  the  event  of  a  declared  disaster  at  the 
production  server  in  Redstone  Arsenal,  Alabama. 
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•  DOT&E  and  ATEC  are  planning  to  conduct  or  participate  in 
a  series  of  test  events  to  address  the  recommended  changes 
listed  in  the  DOT&E -published  IOT&E  report. 

-  Lead  Site  Verification  Test  at  the  National  Guard,  Army 
Reserve,  and  the  Directorate  of  Logistics  module  in 
January  2013. 

-  Test  for  scalability  of  GCSS-Army’s  ability  to  support 
the  entire  Army  when  the  loading  tool  can  be  developed, 
verified,  validated,  and  accredited. 

-  Test  for  future  upgrades  as  the  program  manager  develops 
new  functions.  A  Risk  Assessment  will  determine  the 
scope  of  the  test  in  accordance  with  the  DOT&E -published 
Guidelines  for  Operational  Test  and  Evaluation  of 
Information  and  Business  Systems  memorandum. 

Assessment 

DOT&E  evaluated  GCSS-Army  Release  1.1  as  operationally 

effective,  operationally  suitable,  and  survivable  against  cyber 

threats.  DOT&E  recommended  the  Army  address  the  following. 

•  The  Army  conducted  IOT&E  with  545  users,  compared  to  the 
total  expected  user  population  of  168,000  when  fully  deployed. 

•  The  Army  had  not  developed  GCSS-Army  versions  for  Army 
Reserve  and  National  Guard  units  at  the  time  of  the  IOT&E. 

•  The  20 1 0  National  Defense  Authorization  Act  requires 
financial  audibility  by  2017.  GCSS-Army  had  not  achieved 
this  requirement  at  the  time  of  IOT&E. 


Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  is  making 

progress  implementing  the  previous  recommendations. 

•  FY12  Recommendations. 

1 .  The  Army  needs  to  take  steps  to  achieve  financial 
auditability  no  later  than  2017. 

2.  The  Army  should  continue  to  collect  data  for  computational 
(server  capacity,  storage,  and  bandwidth)  and  human 
factors  (help  desk  responsiveness,  overhead  labor  and 
communication  costs,  and  data  noise)  impacts  of  an 
increased  user  base.  Use  such  data  to  establish  a  pattern 

of  demand  on  the  system,  so  that  future  demand  can  be 
adequately  anticipated  and  resourced  as  more  users  come 
online. 

3.  The  Army  should  conduct  test  and  evaluation  when  the 
software  is  developed  for  Army  Reserve  and  National 
Guard  units  in  accordance  with  the  September  2010 
DOT&E -published  Guidelines  for  Operational  Test 
and  Evaluation  of  Information  and  Business  Systems 
memorandum. 
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Joint  Land  Attack  Cruise  Missile  Defense 
Elevated  Netted  Sensor  System  (JLENS) 


Executive  Summary 

•  The  Joint  Land  Attack  Cruise  Missile  Defense  Elevated 
Netted  Sensor  System  (JLENS)  program  experienced 

a  critical  Nunn-McCurdy  cost  breach  due  to  an  FY12 
budget  decision  to  eliminate  procurement  of  all  production 
systems.  USD(AT&L)  directed  the  Army  to  continue  with 
a  limited  development  program  using  the  two  existing 
JLENS  developmental  orbits,  but  did  not  restore  funding 
for  production  systems.  The  Army  is  continuing  to  plan  and 
execute  a  reduced  JLENS  test  program  to  evaluate  JLENS 
technologies  and  capabilities  as  directed  by  USD(AT&L).  The 
Army  Test  and  Evaluation  Command  (ATEC)  will  conduct 
a  limited  assessment  of  JLENS  operational  capabilities  and 
limitations  in  FY13. 

•  During  the  initial  phases  of  contractor  functional  verification 
and  government  developmental  testing  in  2011,  the  JLENS 
demonstrated  surveillance  and  fire  control  radar  detection 
and  tracking  performance  that  exceeded  system  technical 
specifications  for  fixed-wing  and  cruise  missile  air-breathing 
targets  in  a  controlled  test  environment.  The  system  was 
operated  and  maintained  by  contractor  personnel  during  these 
test  events.  Testing  identified  problems  with  operator  controls 
and  displays,  non-cooperative  target  recognition,  friendly 
aircraft  identification,  and  fire  control  radar  software  and  track 
stability;  these  problems  require  future  improvement. 

•  During  the  developmental  integrated  fire  control  test  phase  in 
April  2012,  an  integrated  JLENS  orbit  supported  a  series  of 
simulated  missile  flight  test  engagements  of  airborne  targets 
with  an  operational  Army  air  defense  missile  system.  This 
phase  concluded  with  a  successful  live  missile  flight  test  that 
destroyed  a  fixed-wing  target  drone  aircraft  in  a  controlled  test 
environment. 

•  The  Army  and  Navy  conducted  a  joint  JLENS  Navy  Integrated 
Fire  Control  -  Counter  Air  (NIFC-CA)  missile  flight  test 
event  at  White  Sands  Missile  Range,  New  Mexico,  in  late 
September  2012.  The  JLENS  provided  integrated  fire  control 
targeting  information  to  a  Navy  Aegis-based  missile  system 
using  Cooperative  Engagement  Capability  (CEC)  datalinks 

to  successfully  engage  and  destroy  a  surrogate  cruise  missile 
aerial  target. 

•  Based  on  data  collected  during  developmental  testing,  JLENS 
system-level  reliability  is  not  meeting  program  reliability 
growth  goals.  Both  software  and  hardware  reliability 
problems  contribute  to  low  system  reliability. 

System 

•  A  JLENS  orbit  consists  of  separate  surveillance  and  fire 
control  radar  systems  individually  mounted  on  74-meter 


tethered  aerostat  balloons  that  operate  at  altitudes  up  to 
10,000  feet  above  mean  sea  level. 

•  A  180,000-pound  mobile  mooring  station  and  tether  system 
is  used  to  launch,  recover,  and  secure  each  aerostat  system. 

The  aerostat  tether  system  provides  radar  control  and  data 
transfer  links  to  supporting  ground  control  and  data  processing 
stations.  JLENS  is  deployable  to  pre-planned  operational  sites 
that  have  been  prepared  to  support  mobile  mooring  station 
operations.  Five  days  are  required  to  transition  between  fully 
operational  status  and  a  transportable  configuration.  Operators 
control  the  radar,  process  data,  and  transmit  radar  track 
information  from  mobile  communication  and  control  stations 
co-located  with  the  mobile  mooring  stations.  A  mobile  power 
generation  and  distribution  system  and  associated  ground 
equipment  support  each  JLENS  orbit. 

Mission 

•  Army  air  defense  units  equipped  with  the  JLENS  provide 
persistent  air  and  missile  threat  warning  to  friendly  forces, 
target  identification,  target  cueing  for  airborne  interceptor 
aircraft,  and  precision  targeting  information  to  ground-based 
air  defense  weapons  systems. 

•  Primary  JLENS  air-breathing  targets  include  all  fixed-  and 
rotary-wing  aircraft,  unmanned  aerial  vehicles,  and  land  attack 
cruise  missiles.  Secondary  targets  include  surface  moving 
targets,  large  caliber  rockets,  and  tactical  ballistic  missiles. 

Major  Contractor 

Raytheon  Integrated  Defense  Systems  -  Andover,  Massachusetts 
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Activity 

•  The  JLENS  program  experienced  a  critical  Nunn-McCurdy 
cost  breach  due  to  an  FY12  budget  decision  to  eliminate 
procurement  of  all  production  systems.  Following  a 
Nunn-McCurdy  review,  USD(AT&L)  rescinded  the  JLENS 
Acquisition  Program  Baseline  and  directed  the  Army  to 
continue  with  a  reduced  JLENS  test  program  using  the  two 
existing  JLENS  developmental  orbits.  The  focus  of  the 
reduced  test  program  is  to  improve  airborne  and  surface 
moving  target  capabilities  in  advance  of  possible  JLENS 
participation  in  an  FY14  Combatant  Commander  exercise. 
USD(AT&L)  did  not  authorize  the  program  to  complete 
the  previously  planned  system  development  program  or  to 
proceed  to  a  Milestone  C  or  production  decision. 

•  Based  on  the  USD(AT&L)  direction,  the  Army  revised 
the  JLENS  operational  test  strategy,  transitioning  from  the 
previously  planned  Milestone  C  operational  assessment  to 
a  limited  Early  User  Test  (EUT)  in  FY13.  The  purpose  of 
the  EUT  event  is  to  assess  JLENS  operational  capabilities 
and  limitations  in  advance  of  the  potential  FY14  Combatant 
Commander  exercise.  During  this  limited  assessment. 

Soldier  operators  will  conduct  missions  with  contractor 
support.  All  system  maintenance  support  will  be  provided 
by  contractor  personnel.  The  Army  has  scheduled  execution 
of  EUT  operational  flight  test  scenarios  for  November  2012, 
with  additional  modeling  and  simulation  events  planned  in 
March  2013.  Since  no  further  JLENS  production  is  currently 
authorized,  a  JLENS  IOT&E  event  will  not  be  required. 

•  The  Army  completed  the  first  phase  of  JLENS  contractor 
functional  verification  testing  and  government  test  phase  in 
December  2011  at  the  Utah  Test  and  Training  Range  (UTTR). 
This  phase  included  contractor  and  government  developmental 
testing  of  the  surveillance  radar  and  fire  control  radars 
installed  on  their  individual  aerostat  systems.  The  primary 
focus  of  these  test  events  was  to  develop  an  initial  capability 
for  the  surveillance  and  fire  control  radars  to  independently 
detect,  identify,  and  track  airborne  targets.  This  phase  also 
provided  initial  information  on  system  reliability. 

•  From  February  through  April  2012,  the  Army  conducted 
a  series  of  developmental  integrated  fire  control  tests 
with  both  radars  operating  as  a  single,  integrated  orbit  to 
support  simulated  air  defense  missile  engagements  using  an 
operational  Army  air  defense  missile  system.  This  test  phase 
concluded  with  a  live  missile  flight  test  engagement  against  a 
target  drone. 

•  The  program  entered  the  second  phase  of  developmental 
testing  in  August  2012.  This  phase  focused  on  improving 
orbit-level  stability  and  technical  maturity  for  airborne  and 
surface  moving  target  detection,  identification,  and  tracking, 
including  operations  in  a  basic  electronic  attack  environment. 
New  capabilities  under  development  in  this  phase  included 
interoperability  with  additional  datalink  systems  such  as  CEC, 
Integrated  Broadcast  Service  (IBS),  Joint  Range  Extension 
Application  Protocol  C  (JREAP  C),  and  Tactical  Voice 
Communications.  The  phase  also  included  Soldier  operator 
and  maintenance  training  activities,  although  contractor 


personnel  continued  to  operate  and  maintain  the  system.  This 
test  phase  was  completed  in  September  2012  and  data  analysis 
is  in  progress. 

•  In  September  2012,  the  JLENS  system  supported  a  NIFC-CA 
Live  Fire  Demonstration  missile  flight  test  event  at  White 
Sands  Missile  Range. 

Assessment 

•  During  the  initial  phases  of  contractor  functional  verification 
and  government  developmental  testing  in  2011,  the  JLENS 
demonstrated  surveillance  and  fire  control  radar  detection 
and  tracking  performance  that  exceeded  system  technical 
specifications  for  fixed-wing  and  cruise  missile  air-breathing 
targets  in  a  controlled  test  environment.  Contractor  personnel 
operated  and  maintained  the  system  during  these  test  events. 

•  Testing  showed  that  the  fire  control  radar  consistently 
provided  fire  control  quality  tracking  data  that  were  sufficient 
to  support  air  defense  missile  engagements.  The  system 
also  demonstrated  a  limited  target  identification  capability 
that  partially  met  requirements  and  basic  interoperability 
with  other  air  defense  systems  using  Link  16  data  transfer 
systems.  Testing  included  a  successful  demonstration  of 

the  fully-deployed  aerostat  tether  system,  including  power 
and  fiber-optic  data  transmission  paths.  The  test  also 
identified  critical  performance  areas  for  improvement  to 
include:  non-cooperative  target  recognition,  friendly  aircraft 
identification  capabilities,  fire  control  radar  software  stability, 
and  target  track  consistency.  During  this  test  phase,  the 
system  was  operated  primarily  by  contractor  personnel  using 
non-production  representative  engineering  control  systems  and 
operator  interfaces. 

•  During  the  developmental  integrated  fire  control  test 
phase  in  April  2012,  an  integrated  JLENS  orbit  supported 
a  series  of  simulated  missile  flight  test  engagements  of 
airborne  targets  with  an  operational  Army  air  defense 
missile  system.  These  tests  verified  that  JLENS  detection 
and  tracking  technical  performance  continued  to  mature 
and  were  sufficient  to  support  basic  air  defense  missile 
engagements.  This  phase  concluded  with  a  successful  live 
missile  flight  test  that  destroyed  a  target  drone  aircraft  in  a 
controlled  test  environment.  During  this  phase,  the  JLENS 
system  was  operated  primarily  by  contractor  personnel  using 
non-production  representative  engineering  control  systems  and 
operator  interfaces. 

•  During  the  joint  JLENS  NIFC-CA  missile  flight  test  at  White 
Sands  Missile  Range,  the  JLENS  provided  integrated  fire 
control  targeting  information  to  a  Navy  Aegis-based  missile 
system  using  CEC  datalinks  to  successfully  engage  and 
destroy  a  surrogate  cruise  missile  aerial  target. 

•  Based  on  data  collected  during  developmental  testing,  JLENS 
system-level  reliability  is  not  meeting  program  growth  goals. 
At  the  beginning  of  the  DT-2  test  phase  in  August  2012,  the 
Army  assessed  orbit  level  reliability  to  be  21  hours  Mean 
Time  Between  System  Abort  (MTBSA),  well  below  the 

108  hours  MTBSA  necessary  to  meet  operational  reliability 
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and  availability  requirements.  Both  software  and  hardware 
reliability  problems  contribute  to  low  system  reliability. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 


•  FY12  Recommendations.  The  Army  should: 

1.  Complete  the  planned  JLENS  EUT  and  provide  an 
operational  capabilities  and  limitations  report  prior  to 
initiating  a  Combatant  Command  exercise. 

2.  Develop  a  reliability  improvement  plan  to  address  poor 
system-level  reliability  prior  to  a  JLENS  fielding  decision. 
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Joint  Light  Tactical  Vehicle  (JLTV) 


AM  General  JLTV 


Lockheed  Martin  Systems  JLTV 


Executive  Summary 

•  DOT&E  approved  the  Joint  Light  Tactical  Vehicle  (JLTV) 

Test  and  Evaluation  Master  Plan  (TEMP)  in  July  2012.  The 
TEMP  addresses  the  test  and  evaluation  strategy  to  support  the 
selection  of  a  single  vendor  to  produce  JLTV  low-rate  initial 
production  vehicles,  and  includes  a  reliability  growth  plan  and 
adequate  operational  and  live  fire  testing. 

•  In  August  2012  the  USD(AT&L)  granted  approval  for  the 
program  to  enter  Engineering  Manufacturing  and  Development 
(EMD)  at  Milestone  B. 

•  Based  on  JLTV  prototype  performance  in  Technology 
Development  (TD)  phase  tests,  demonstrating  the  reliability 
threshold  requirement  of  2,400  Mean  Miles  Between 
Operational  Mission  Failure  is  dependent  on  the  vendors 
designing  reliability  into  vehicles  and  successfully  integrating 
government-furnished  mission  equipment  in  the  vehicles. 

•  The  Army  underbody  threat  size  requirements  are  equivalent 
to  those  of  the  Mine  Resistant  Ambush  Protected  (MRAP) 
All-Terrain  Vehicle  (M-ATV)  without  an  underbody 
improvement  kit.  The  ability  to  achieve  this  level  of  protection 
while  satisfying  other  JLTV  requirements  is  not  yet  known. 

•  The  program  awarded  contracts  to  Oshkosh  Corporation, 

AM  General,  and  Lockheed  Martin  for  the  EMD  phase  of 
the  JLTV  Family  of  Vehicles  (FoV)  program  in  August  2012. 
The  program  has  allotted  33  months  for  the  EMD  phase.  The 
contractors  will  deliver  22  full-up  prototypes  per  contractor 
in  the  12  months  after  the  EMD  contract  is  awarded  for  the 
14-month  government  test  program. 

System 

•  The  JLTV  FoV  is  the  Marine  Corps  and  Army  partial 
replacement  for  the  High  Mobility  Multi-purpose  Wheeled 
Vehicle  (HMMWV).  The  Services  intend  the  JLTV  to  provide 
increased  crew  protection  against  IED  and  underbody  attacks, 
improved  mobility,  and  higher  reliability  than  the  HMMWV. 

•  The  JLTV  FoV  consists  of  two  vehicle  categories:  the 
JLTV  Combat  Tactical  Vehicle  (CTV),  designed  to  seat  four 
passengers;  and  the  JLTV  Combat  Support  Vehicle  (CSV), 
designed  to  seat  two  passengers. 

•  The  JLTV  CTV  has  a  3,500-pound  payload  and  three  mission 
package  configurations: 

-  Close  Combat  Weapons  Vehicle 

-  General  Purpose  Vehicle 

-  Heavy  Guns  Carrier  Vehicle 


Oshkosh  Defense  JLTV 


•  The  JLTV  CSV  has  a  5,100-pound  payload  and  two  mission 
package  configurations: 

-  Utility  Prime  Mover 

-  Shelter  Carrier 

•  The  JLTV  program  is  using  a  competitive  acquisition 
strategy.  During  the  EMD  phase,  the  program  will  test  and 
assess  at  least  three  vendors’  FoVs. 

Mission 

•  Military  units  employ  JLTV  as  a  light  tactical  wheeled 
vehicle  to  support  all  types  of  military  operations.  JLTVs 
will  be  used  by  airborne,  air  assault,  light,  Stryker,  and 
heavy  forces  as  reconnaissance,  maneuver,  and  maneuver 
sustainment  platforms. 

•  Small  ground  combat  units  employ  JLTV  in  combat  patrols, 
raids,  long-range  reconnaissance,  and  convoy  escort. 

Major  Contractors 

•  Oshkosh  Corporation  -  Oshkosh,  Wisconsin 

•  AM  General  -  South  Bend,  Indiana 

•  Lockheed  Martin  Corporation  -  Dallas,  Texas 
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Activity 

•  The  program  completed  TD  phase  testing  in  4QFY 1 1  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Strategy  (TES). 

•  In  January  2012,  the  program  released  the  Request  for 
Proposal  to  industry  to  solicit  vendor  participation  in  the  JLTV 
EMD  phase. 

•  The  Joint  Requirements  Oversight  Council  approved  the  JLTV 
Capability  Development  Document  and  designated  the  Army 
as  lead  Service  for  the  program  in  March  2012. 

•  DOT&E  approved  the  JLTV  TEMP  in  July  2012.  The  TEMP 
addresses  the  TES  to  support  the  selection  of  a  single  vendor 
to  produce  JLTV  low-rate  initial  production  vehicles,  and 
includes  a  reliability  growth  plan  and  adequate  operational  and 
live  fire  testing. 

•  The  USD(AT&L)  approved  Milestone  B  in  August  2012, 
which  permitted  the  program  to  enter  EMD. 

•  The  program  awarded  contracts  to  Oshkosh  Corporation, 

AM  General,  and  Lockheed  Martin  for  the  EMD  phase  of  the 
JLTV  FoV  program  in  August  2012.  The  program  has  allotted 
33  months  for  the  EMD  phase.  The  contractors  will  deliver 
22  full-up  prototypes  per  contractor  in  the  12  months  after  the 
EMD  contract  is  awarded  for  the  14-month  government  test 
program. 


Assessment 

•  Based  on  JLTV  prototype  performance  in  TD  phase  tests, 
demonstrating  the  reliability  requirement  of  2,400  Mean 
Miles  Between  Operational  Mission  Failure  during  EMD 
is  dependent  on  the  vendors  designing  reliability  into  their 
vehicles  and  successfully  integrating  government-furnished 
mission  equipment  in  the  vehicles. 

•  The  Army  underbody  threat  size  requirements  are  equivalent 
to  the  threat  size  requirements  of  the  M-ATV  without  an 
underbody  improvement  kit.  The  ability  to  achieve  this  level 
of  protection  while  satisfying  other  JLTV  requirements  is  not 
yet  known. 

•  The  planned  EMD  reliability  growth  testing  and  corrective 
action  periods  have  limited  time  to  identify  and  resolve  failure 
modes  prior  to  initiation  of  the  Limited  User  Test. 

•  The  Limited  User  Test  and  live  fire  test  programs  have 
adequate  quantities  and  variants  of  the  JLTV  prototypes  to 
assess  the  JLTV  FoV  operational  capabilities  and  survivability. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  addressed 
all  previous  recommendations. 

•  FY12  Recommendations.  None. 
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MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  Army  conducted  the  Gray  Eagle  IOT&E  at  Edwards  AFB, 
California,  and  the  National  Training  Center  (NTC),  Fort 
Irwin,  California,  July  30  through  August  17,  2012. 

•  The  Army  conducted  the  IOT&E  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

•  DOT&E  is  completing  a  Beyond  Low-Rate  Initial 
Production  (BLRIP)  report  supporting  the  Gray  Eagle 
Full-Rate  Production  decision  planned  for  April  2013.  In 
that  report,  DOT&E  concludes  the  Gray  Eagle-equipped 
unit  was  effective  at  operating  the  MQ-1C  system  and  has 
the  potential  to  provide  effective  support  to  combat  units, 
but  the  Army  needs  to  continue  to  develop  the  tactics, 
techniques,  and  procedures;  the  training;  and  the  doctrine 
required  to  effectively  integrate  this  capability  into  combat 
operations.  The  Gray  Eagle  system  is  operationally  suitable. 
The  Gray  Eagle  meets  its  crew  force  protection  survivability 
requirements  by  providing  up-armored  capability  to  the  cab  of 
the  vehicles  transporting  the  Ground  Control  Stations  during 
company  movement.  The  Gray  Eagle  aircraft  is  not  survivable 
in  a  mid-  to  high-threat  environment. 

System 

The  Gray  Eagle  Unmanned  Aircraft  System  (UAS)  is  composed 

of  the  following  major  components: 

•  Twelve  unmanned  aircraft  each  with  a  Common  Sensor 
Payload  (CSP)  with  an  electro-optical/infrared  with  a  Laser 
Range  Finder/Laser  Designator  capability,  a  STARLite 
Synthetic  Aperture  Radar/Ground  Moving  Target  Indicator 
(SAR/GMTI)  sensor  payload,  and  an  Air  Data  Relay  (ADR) 
control  capability 

•  Each  aircraft  is  equipped  with  a  Standard  Equipment  Package 
that  includes  a  communications  relay  package,  Identification 
Friend-or-Foe  equipment,  and  Air  Traffic  Control  radios 

•  Each  aircraft  has  the  ability  to  carry  up  to  four  Hellfire  P+ 
missiles 

•  Five  Ground  Control  Stations  designated  as  the  One  System 
Ground  Control  Station  (OSGCS) 


•  Five  Tactical  Common  Datalinks  (TCDL)  Ground  Data 
Terminals 

•  One  Satellite  Communications  (SATCOM)  Ground  Data 
Terminal 

•  Six  SATCOM  Air  Data  Terminals 

•  Two  Automatic  Take-off  and  Landing  Systems,  which  consist 
of  four  Take-off  and  Landing  System-Tracking  Systems  and 
antennas 

Mission 

Commanders  employ  Gray  Eagle  Companies  to  execute 
Reconnaissance,  Surveillance,  Security,  Attack,  and  Command 
and  Control  missions  in  support  of  assigned  Division  Combat 
Aviation  Brigade,  Fires  Brigade,  Battlefield  Surveillance 
Brigade,  Brigade  Combat  Teams,  and  other  Army  and  Joint  Force 
units  based  upon  the  Division  Commander’s  mission  priorities. 

Major  Contractor 

General  Atomics  Aeronautical  Systems,  Inc.,  Aircraft  Systems 
Group  -  Poway,  California 


Activity 

•  The  Gray  Eagle  UAS  participated  in  the  Apache  Block  3 
IOT&E  in  March  through  April  2012,  at  the  NTC  in  Fort 
Irwin,  California,  to  conduct  Manned-Unmanned  Teaming 
operations.  A  single  Gray  Eagle  UAS  with  associated  flight 
crew  and  personnel  provided  mission  support  to  the  IOT&E 
from  Edwards  AFB,  California. 

•  In  July  2012,  and  as  a  result  of  the  direction  received  from  the 
Low-Rate  Initial  Production  (LRIP)  III  Acquisition  Decision 


Memorandum,  the  project  manager  for  UAS  requested  the 
Army  Aviation  Center  seek  a  deferral  of  the  requirement  to 
meet  the  Reliability  Key  System  Attributes  (KSAs)  and  the 
Sustainment  Key  Performance  Parameter  (KPP)  from  IOT&E 
to  FOT&E.  The  Army  deferred  the  requirement  of  meeting 
the  KSAs  on  August  16,  2012.  The  Joint  Requirements 
Oversight  Council  approved  the  deferral  of  meeting  the  KPP 
on  November  16,  2012. 
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•  The  Army  conducted  the  Gray  Eagle  IOT&E  at  Edwards 
AFB,  California,  and  the  NTC,  Fort  Irwin,  California, 

July  30  through  August  17,  2012,  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan.  The  Army  based  the  IOT&E  unit  at  Edwards  AFB,  and 
from  there  it  conducted  missions  in  support  of  the  Brigade 
Combat  Team  conducting  a  training  rotation  at  the  NTC 
approximately  130  kilometers  away.  The  IOT&E  unit  flew 
1,090  flight  hours  and  conducted  missions  at  operational 
ranges  exceeding  150  kilometers.  The  IOT&E  unit  routinely 
flew  aircraft  between  8,000  and  13,000  feet  mean  sea  level 
(MSL)  with  a  maximum  altitude  being  flown  of  24,500  feet 
MSL. 

•  There  were  eight  Flellfire  missile  engagements  during  IOT&E. 
Six  were  cooperative  engagements  with  Apache  helicopters, 
meaning  one  aircraft  provided  the  lasing  of  the  target  while 
the  other  aircraft  launched  the  missile.  Two  engagements 
were  autonomous,  meaning  the  Gray  Eagle  provided  the  lasing 
of  the  target  and  launched  the  missile  on  its  own.  All  eight 
missile  engagements  hit  the  intended  target. 

•  DOT&E  is  completing  a  BLRIP  report  supporting  the  Gray 
Eagle  Full-Rate  Production  decision  planned  for  April  2013. 

Assessment 

•  During  IOT&E,  the  Gray  Eagle-equipped  unit  was  effective  at 
operating  the  MQ-1C  system  and  demonstrated  the  potential 
to  provide  effective  support  to  combat  units,  but  the  Army 
has  not  effectively  integrated  this  capability  into  combat 
operations.  Army  integration  of  Gray  Eagle  into  employment 
concepts,  and  development  of  tactics,  techniques,  and 
procedures  are  not  mature  and  training  afforded  to  the  IOT&E 
unit  before  the  test  was  not  complete.  Soldiers  did  not  receive 
training  on  fundamentals  of  reconnaissance,  mission  planning, 
set-up  and  operation  of  radios,  distribution  of  video,  STARLite 
SAR/GMTI  capabilities  and  operation,  or  employment  of  Gray 
Eagle. 

•  The  Gray  Eagle  system  has  more  capability  and  functionality 
today  than  it  demonstrated  in  previous  operational  tests. 
Increases  in  capability  demonstrated  during  the  2012  IOT&E 
include: 

-  The  CSP  providing  the  electro-optical/infrared  with  a  Laser 
Range  Finder/Laser  Designator  capability 

-  A  STARLite  SAR/GMTI  payload 

-  The  ability  to  conduct  aircraft  operations  via  encrypted 
TCDL,  SATCOM  datalink,  as  well  as  the  ADR  aircraft 
control  capability 

•  The  IOT&E  unit  completed  223  of  307  attempted  missions 
during  test,  resulting  in  a  mission  success  rate  of  73  percent. 

•  The  Gray  Eagle  system  is  operationally  suitable.  During 
IOT&E,  the  Gray  Eagle  unit  demonstrated  its  ability  to 
meet  its  operational  tempo  requirement  to  provide  three 
simultaneous  and  continuous  missions  (24-hour  continuous 
reconnaissance,  24-hour  continuous  armed  reconnaissance, 
and  two  5-hour  attack  missions  in  a  24-hour  period).  The 
Gray  Eagle  system  demonstrated  a  Combat  Availability  of 
8 1  percent,  exceeding  the  Sustainment  KPP  requirement 
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of  80  percent.  The  unit  achieved  the  Combat  Availability 
requirement  in  spite  of  failing  to  meet  its  reliability 
requirements.  The  IOT&E  demonstrated  that  the  modeling 
assumptions  that  established  the  reliability  requirements 
thresholds  were  not  valid.  As  a  result,  the  Army  is  reassessing 
whether  or  not  the  reliability  KSAs  need  to  be  changed.  Those 
modified  reliability  requirements,  if  made,  are  planned  to  be 
tested  in  FOT&E. 

•  During  the  IOT&E,  the  Gray  Eagle  demonstrated  KSA  Mean 
Time  Between  System  Abort  (MTBSA)  compared  to  the 
deferred  MTBSA  requirements  of  44  hours  versus  150  hours 
for  the  OSGCS,  55  hours  versus  100  hours  for  the  aircraft, 

218  hours  versus  500  for  the  CSP,  and  97  hours  versus 

500  hours  for  the  STARLite  SAR/GMTI. 

•  The  Gray  Eagle  Company  depended  heavily  on  the 
maintenance  expertise  of  Contractor  Field  Service 
Representatives. 

•  During  the  IOT&E,  remote  video  from  the  Gray  Eagle  to  the 
One  System  Remote  Video  Terminal  (OSRVT)  was  generally 
not  available,  not  clear,  and  not  reliable.  Integration  of  the 
Gray  Eagle  with  a  reliable  remote  video  display  system  is  not 
complete. 

•  The  IOT&E  unit  demonstrated  effective  target  detection  and 
recognition  capability  using  the  electro-optical/infrared  sensor 
with  Laser  Range  Finder/Designator. 

•  The  Flellfire  P+  missile  is  fully  integrated  into  the  Gray  Eagle 
system  when  using  the  TCDL  and  SATCOM  datalinks.  The 
Army  has  not  demonstrated  Flellfire  engagements  via  the  ADR 
datalink  in  developmental  or  operational  testing. 

•  The  Automatic  Take-off  and  Landing  System  worked  as 
designed. 

•  The  Gray  Eagle  is  vulnerable  to  computer  network  attack. 

•  The  Gray  Eagle  meets  its  crew  force  protection  survivability 
KPP  requirement  by  providing  up-armored  capability  to  the 
cab  of  the  vehicles  transporting  the  ground  control  stations 
during  company  movement.  Testing  indicates  that  the  Gray 
Eagle  aircraft  is  not  survivable  in  a  mid-  to  high-threat 
environment.  The  aircraft  can  be  detected  at  operational 
altitudes  visually,  acoustically,  by  late-model  man-portable  air 
defense  systems,  and  by  threat  radar  systems. 

•  The  operator’s  manual  is  not  current  and  in  some  cases  not 
accurate. 

•  The  design  of  the  OSGCS  shelter  has  a  number  of  features 
that  reduce  operator  efficiency  and  increase  operator  stress  and 
fatigue:  volume  control  of  radios,  OSGCS  start-up  procedures, 
procedures  for  operators  to  establish  SATCOM  and  ADR 
datalinks,  work  station  climate  control,  and  poor  ergonomics 
of  the  OSGCS  operator’s  joy  stick  controller. 

-  Operators  are  not  able  to  control  the  volume  on  any  of  the 
radios  within  the  OSGCS.  On  numerous  occasions  during 
missions,  the  air  traffic  control  radio  calls  would  drown  out 
the  operator’s  ability  to  hear  other  communications  on  the 
tactical  radio  networks. 

-  The  OSGCS  start-up  procedures  entail  191  checklist  steps 
and  require  up  to  2  hours  to  execute. 
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-  Procedures  for  operators  to  establish  SATCOM  and  ADR 
datalinks  are  numerous  and  complex  in  nature.  If  the 
operator  does  not  follow  the  prescribed  sequence,  they 
have  to  re-execute  the  procedure  in  its  entirety,  taking  up  to 
45  additional  minutes  to  complete  the  task. 

-  The  computers  and  avionics  within  the  OSGCS  require 
air  conditioning  to  maintain  normal  operation.  Air 
conditioning  controls  have  been  improved;  however,  in 
order  to  stay  warm,  OSGCS  operators  continue  to  wear 
hats,  gloves,  and  cold-weather  gear  even  in  hot  weather 
environments. 

-  Operator  controls  are  not  efficient.  OSGCS  employs  a 
joystick  that  has  no  triggers  or  buttons  that  would  allow 
one-handed  control  of  the  payload  or  aircraft.  Both 
hands  are  required  for  many  basic  tasks  as  the  operator 
provides  inputs  to  both  the  joystick  and  the  keyboard  while 
operating  the  system.  A  cyclic-type  joystick,  such  as  those 
found  in  Army  helicopters,  would  allow  for  one-handed 
multifunction  operation  of  the  system. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 

four  of  the  seven  previous  recommendations.  Outstanding 

previous  recommendations  include: 

1 .  The  Army  should  develop,  optimize,  and  publish 
standardized  procedures  for  the  OSRVT  terminal. 

2.  The  Army  should  revise  and  expand  the  training  program 
and  update  the  operator’s  manual. 

3.  The  Army  should  improve  the  Ground  Control  Shelter 
design. 

•  FY 12  Recommendations. 

1.  The  Army  should: 


-  Continue  to  develop  doctrine;  employment  concepts;  and 
tactics,  techniques,  and  procedures  to  fully  integrate  the 
Gray  Eagle  unit  into  combat  operations. 

-  Train  operators  on  fundamentals  of  reconnaissance, 
mission  planning,  set-up  and  operation  of  radios, 
distribution  of  video,  and  optimal  employment  of  the 
Gray  Eagle. 

-  Continue  to  develop  and  publish  standardized  procedures 
for  distribution  of  Gray  Eagle  video  to  OSRVT  and 
institute  training  across  the  Army  to  facilitate  integration 
of  Gray  Eagle  into  supported  unit  operations. 

-  Revise  and  expand  the  training  program  for  operators  and 
update  the  operator’s  manual. 

-  Modify  the  personnel  plan  to  retain  or  offset 
the  anticipated  loss  of  Contractor  Field  Service 
Representative  support. 

-  Refine  and  train  procedures  for  collection  and 
exploitation  of  SAR/GMTI  imagery. 

2.  The  Product  Office  should: 

-  Simplify,  and  to  the  greatest  extent  possible,  automate 
routine  operator  tasks.  The  2-hour,  191  checklist  steps 
required  to  start  the  Ground  Control  Station  should  be 
streamlined  and  be  made  less  susceptible  to  operator 
errors. 

-  Simplify  procedures  for  operators  to  establish  SATCOM 
and  ADR  datalinks. 

-  Qualify  the  ADR  datalink  for  employment  of  Hellfire 
missiles. 

-  Improve  OSGCS  functionality  by  increasing  operator 
control  of  radio  volume,  temperature,  and  joy  stick 
functionality. 
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Paladin  Integrated  Management  (PIM) 


Executive  Summary 

•  DOT&E  approved  the  Paladin  Integrated  Management  (PIM) 
Engineering  and  Manufacturing  Development  Test  and 
Evaluation  Master  Plan  on  March  5,  2012. 

•  In  1QFY12,  PIM  completed  Phase  I  government 
developmental  testing  with  five  prototype  howitzers 
and  two  prototype  ammunition  carriers.  The  howitzer 
reliability  estimate  from  Phase  I  reliability,  availability,  and 
maintainability  (RAM)  missions  was  45  hours  Mean  Time 
Between  System  Abort  (MTBSA)  compared  to  a  requirement 
of  62  hours. 

•  In  July  2012,  PIM  vehicles  commenced  Phase  II 
developmental  testing  in  preparation  for  a  Limited  User  Test 
scheduled  for  October  31  through  November  8,  2012,  to 
support  a  June  2013  Milestone  C  decision. 

•  Testing  to  characterize  the  protection  provided  by  Ml 09 
Family  of  Vehicles  (FoV)  armor  configurations  began  in 
August  2012  and  is  scheduled  to  continue  through  the  end  of 
the  year. 

•  In  2011,  DOT&E  raised  concerns  regarding  the  M109  FoV 
force  protection  and  survivability  requirements.  As  a  result, 
the  Army  improved  the  systems’  ballistic  vulnerability 
requirements,  and  OSD  directed  the  Army  to  design,  develop, 
and  test  an  underbelly  kit  to  address  operationally-relevant 
underbody  blast  threats. 

•  OSD  also  directed  the  Army  to  characterize  the  expected 
impact  of  the  objective-level  underbelly  kit  on  howitzer 
performance  and  vehicle  RAM  by  integrating  such  a  kit  into 
the  PIM  program’s  development  and  testing,  and  planned 
Production  Qualification  Testing. 

•  The  program’s  current  plans  are  schedule  driven,  with  limited 
time  for  correction  of  deficiencies  identified  in  developmental 
testing  and  little  flexibility  with  individual  component  test 
schedules.  The  program  must  wait  for  the  production  of 
low-rate  initial  production  (LR1P)  vehicles  before  verifying 
most  corrective  actions. 

System 

•  The  M109  FoV  PIM  consists  of  two  vehicles,  the 
Self-Propelled  Howitzer  (SPH)  and  Carrier,  Ammunition, 
Tracked  (CAT). 

•  The  M109  FoV  SPH  is  a  tracked,  self-propelled  155  mm 
howitzer  designed  to  improve  sustainability  over  the  legacy 


M109A6  howitzer  fleet.  The  full-rate  production  howitzers 
will  have  a  newly  designed  hull,  modified  Bradley  Fighting 
Vehicle  power  train  and  suspension,  and  a  high-voltage 
electrical  system.  Operated  by  a  crew  of  4,  the  howitzer  is 
capable  of  achieving  ranges  of  22  kilometers  using  standard 
projectiles  and  30  kilometers  using  rocket-assisted  projectiles. 

•  The  M109  FoV  CAT  supplies  the  howitzer  with  ammunition. 
The  full-rate  production  ammunition  carriers  will  have  a 
common  chassis  with  the  SPH.  The  ammunition  carriers  are 
designed  to  carry  12,000  pounds  of  ammunition  in  various 
configurations  and  a  crew  of  4  Soldiers. 

•  The  Army  intends  to  employ  the  M109  FoV  as  part  of  a  Fires 
Battalion  in  the  Armored  Brigade  Combat  Team  and  Artillery 
Fires  Brigades  with  the  capability  to  support  any  Brigade 
Combat  Team. 

•  The  Army  plans  to  field  580  sets  of  the  M109  FoV  with 
full-rate  production  vehicles  beginning  in  FY17. 

Mission 

Field  Artillery  units  will  use  the  M109  FoV  to  destroy,  neutralize, 
or  suppress  the  enemy  by  providing  massed  and  precision 
indirect  fire  effects  in  support  of  maneuver  units  in  multiple 
levels  of  conflict  to  include  major  combat  operations  and 
irregular  warfare. 

Major  Contractor 

BAE  Systems  -  York,  Pennsylvania 


Activity 

•  In  1QFY12,  PIM  completed  Phase  I  government  Center,  Arizona,  included  five  RAM  missions  (firing  223 

developmental  testing  with  five  prototype  howitzers  and  two  rounds  and  moving  17.4  miles  per  firing  mission  -  equivalent 

prototype  CAT  platforms.  Phase  I  SPH  testing  at  Yuma  Test 
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to  1  combat  day  at  the  Paladin  operational  tempo),  firing 
performance  tests. 

•  Testers  at  the  Aberdeen  Test  Center,  Maryland,  conducted 
Phase  1  testing  of  both  vehicles  to  determine  compliance  with 
performance  specifications.  A  2,400-mile  demonstration 
with  the  CAT  was  terminated  early  because  of  a  transmission 
failure. 

•  In  2011,  DOT&E  raised  concerns  regarding  the  M109  FoV 
force  protection  and  survivability  requirements.  As  a  result, 
the  Army  improved  the  systems’  ballistic  vulnerability 
requirements,  and  OSD  directed  the  Army  to  design,  develop, 
and  test  an  underbelly  kit  to  address  operationally-relevant 
underbody  blast  threats. 

•  OSD  also  directed  the  Army  to  characterize  the  expected 
impact  of  the  objective-level  underbelly  kit  on  howitzer 
performance  and  vehicle  RAM  by  integrating  such  a  kit  into 
the  PIM  program’s  development  and  testing,  and  planned 
Production  Qualification  Testing. 

•  DOT&E  approved  the  PIM  Test  and  Evaluation  Master  Plan 
on  March  5,  2012. 

•  Prototype  SPHs  completed  refurbishment  in  June  2012,  and 
entered  Phase  II  government  developmental  testing,  which 
completes  production  prove-out  testing  with  prototype 
vehicles.  Phase  II  included  an  additional  10  RAM  missions 
using  the  Heavy  Brigade  Combat  Team  operational  tempo,  the 
logistics  demonstration,  and  software  verification.  Phase  II 
completed  in  mid-October  2012. 

•  Testing  to  characterize  the  protection  provided  by  M109  FoV 
armor  configurations  began  in  August  2012  and  is  scheduled  to 
continue  through  the  end  of  the  calendar  year. 

•  The  Army  has  scheduled  a  PIM  Limited  User  Test  for 
October  31  through  November  8,  2012,  to  support  a 
Milestone  C  decision  in  June  2013.  The  IOT&E  is  scheduled 
for4QFY16. 

Assessment 

•  The  current  program  is  schedule-driven,  with  limited  time  for 
correction  of  deficiencies  identified  in  developmental  testing 
and  little  flexibility  in  schedules  for  individual  tests.  Most 
corrective  actions  must  wait  for  testing  with  LRIP  vehicles  to 
verify  the  corrective  action. 

•  The  howitzer  reliability  estimate  from  Phase  I  RAM  missions, 
45  hours  MTBSA,  is  the  initial  point  on  a  reliability  growth 


curve  designed  to  grow  vehicle  reliability  above  the  62-hour 
MTBSA  needed  to  meet  the  reliability  requirement. 

•  As  Phase  I  RAM  testing  progressed,  vehicle  discrepancies 
increased  as  PIM  subsystems  had  problems  withstanding 
repeated  gun  shock.  Legacy  subsystems,  including  the  Paladin 
Digital  Fire  Control  System  (PDFCS),  demonstrated  fewer 
gun-shock  generated  discrepancies. 

•  The  PDFCS  experienced  frequent  failures  attributed  to 
the  Muzzle  Velocity  Radar  System  (MVRS).  A  software 
modification  corrected  a  timing  problem  between  MVRS  and 
PDFCS,  which  is  believed  to  have  caused  those  failures. 

•  In  Phase  I  testing,  the  PIM  prototype  SPH  demonstrated 
the  ability  to  conduct  conventional  fire  missions,  and 
verify  compatibility  with  current  155  mm  ammunition.  In 
early  testing  with  the  initial  transmission,  the  SPH  failed 
to  meet  the  climbing  requirement.  Subsequent  testing  has 
not  demonstrated  the  ability  to  meet  this  requirement  with 
confidence  using  correct  fluid  levels  per  the  Technical  Manual. 

•  Vehicle  deficiencies  were  identified  during  Phase  II  RAM 
testing  with  the  automotive  subsystems  of  the  SPH.  These 
problems  prevented  the  vehicle  from  conducting  required 
short-duration  survivability  movements  in  a  timely  manner. 

•  After  the  critical  design  review,  the  program  manager 
identified  a  number  of  corrective  action,  producibility,  and 
obsolescence  (CPO)  engineering  changes  to  the  PIM  design 
that  will  be  implemented  between  LRIP  and  the  IOT&E.  The 
schedule  for  development,  test,  and  implementation  of  those 
CPO  changes  is  high-risk  and  challenging. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Provide  an  LRIP-configured  prototype  as  early  as  possible 
to  continue  characterizing  armor  performance,  address 
critical  ballistic  vulnerability  data  voids,  and  provide  time 
for  implementation  of  corrective  actions  before  entering 
full-up  system-level  testing. 

2.  Ensure  that  CPO  changes  are  fully  qualified  in  system-level 
testing  rather  than  relying  solely  on  component-level 
qualification  before  implementing  them  in  the  LRIP 
vehicles  scheduled  for  the  IOT&E. 
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Patriot  Advanced  Capability-3  (PAC-3) 


Executive  Summary 

•  The  Army  began  the  Post-Deployment  Build-7  (PDB-7) 
Limited  User  Test  (LUT)  operational  test  in  FY12.  The 
PDB-7  LUT  included  hardware-in-the-loop  (HWIL)  testing, 
sustained  operations  testing,  and  a  combined  missile  flight  test 
that  consolidated  three  individual  missile  flight  tests  into  one 
campaign. 

•  The  Army  conducted  five  major  developmental  Patriot 
flight  test  missions  and  the  PDB-7  Developmental  Test  and 
Evaluation  (DT&E)  in  FY12. 

•  The  Missile  Defense  Agency  conducted  an  integrated  flight 
test  of  the  Ballistic  Missile  Defense  System  (BMDS)  in 
October  2012,  during  which  Patriot  engaged  and  killed  a 
cruise  missile  target  and  a  tactical  ballistic  missile  target  in  the 
debris  field  caused  by  another  BMDS  intercept. 

•  In  the  seven  U.S.  developmental  and  operational  flight  tests 
conducted  in  FY12,  Patriot  achieved  successful  intercepts  of 
six  short-range  ballistic  missile  targets  and  five  air-breathing 
threat/cruise  missile  targets  using  Patriot  Advanced 
Capability-3  (PAC-3)  and  Guidance  Enhanced  Missile  (GEM) 
missiles. 

•  The  Army  also  conducted  five  flight  tests  for  an  international 
Foreign  Military  Sales  (FMS)  customer,  during  which  Patriot 
intercepted  four  of  five  tactical  ballistic  missile  targets  and 
three  of  three  air-breathing  threat/cruise  missile  targets.  The 
fifth  FMS  flight  test  concurrently  successfully  fulfilled  a  long 
standing  PAC-3  Engineering  Manufacturing  and  Development 
phase  requirement. 

System 

•  Patriot  is  a  mobile  air  and  missile  defense  system  that 
counters  missile  and  aircraft  threats.  The  system  includes  the 
following: 

-  C-band  phased-array  radars  for  detecting,  tracking, 
classifying,  identifying,  and  discriminating  targets 

-  Battalion  and  battery  battle  management  elements 

-  Communications  Relay  Groups  and  Antenna  Mast  Groups 
for  communicating  between  battery  and  battalion  assets 

-  A  mix  of  PAC-3  hit-to-kill  missiles  and  PAC-2  blast 
fragmentation  warhead  missiles  for  negating  missile  and 
aircraft  threats 

•  The  newest  version  of  the  PAC-3  missile  is  the  Cost  Reduction 
Initiative  (CRI)  missile.  In  addition,  the  Army  is  developing 


the  PAC-3  Missile  Segment  Enhancement  (MSE)  missile 
with  increased  battlespace  defense  capabilities  and  improved 
lethality. 

•  Earlier  versions  of  Patriot  missiles  include  the  Patriot  Standard 
missile,  the  PAC-2  Anti-Tactical  Missile,  and  the  GEM  family 
(includes  the  GEM-T  and  GEM-C  missile  variants  intended 

to  counter  tactical  ballistic  missiles  and  cruise  missiles, 
respectively). 

•  DoD  intended  the  Medium  Extended  Air  Defense  System 
(MEADS)  to  replace  the  Patriot  system.  The  DoD  has  decided 
not  to  field  MEADS  and  plans  to  conclude  the  design  and 
development  phase  of  the  program  in  FY13. 

Mission 

Combatant  Commanders  use  Patriot  to  defend  deployed  forces 
and  critical  assets  from  missile  and  aircraft  attack  and  to 
defeat  enemy  surveillance  air  assets  (such  as  unmanned  aerial 
vehicles)  in  all  weather  conditions,  and  in  natural  and  induced 
environments. 

Major  Contractors 

•  Raytheon  Integrated  Defense  Systems  -  Tewksbury, 
Massachusetts 

•  Lockheed  Martin  Missile  and  Fire  Control  -  Grand  Prairie, 
Texas 


Activity 

•  In  accordance  with  the  DOT&E-approved  Test  and 
Evaluation  Master  Plan,  the  Army  began  the  PDB-7  LUT 
on  May  31,  2012,  at  White  Sands  Missile  Range  (WSMR), 


New  Mexico.  The  PDB-7  LUT  is  expected  to  end  in 
November  2012  with  the  completion  of  the  PDB-7  LUT 
regression  test. 
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•  The  Army  conducted  the  PDB-7  LUT  operational  missile  flight 
test  (P7L-1/2/3)  at  WSMR  in  August  2012.  During  this  test, 
Patriot: 

-  Engaged  and  intercepted  one  tactical  ballistic  missile  target 
with  a  ripple  launch  (firing  of  missiles  in  quick  succession) 
of  GEM-T/PAC-3  CRI  missiles. 

-  Engaged  a  second  tactical  ballistic  missile  target  with 
a  ripple  launch  of  two  PAC-3  missiles.  This  second 
tactical  ballistic  missile  target  self-destructed  before  the 
interceptors  reached  it;  therefore,  the  endgame  segment 
of  the  second  tactical  ballistic  missile  engagement  was 
deemed  a  “No  Test.” 

-  Engaged  and  intercepted  a  cruise  missile  target  with  a 
GEM-T  missile  in  the  debris  field  resulting  from  the 
destruction  of  the  two  tactical  ballistic  missile  targets. 

•  The  Army  conducted  the  PDB-7  DT&E  at  WSMR  from 
July  2011  to  March  2012. 

-  During  PDB-7  flight  test  P7-4  in  November  2011,  Patriot 
engaged  a  short-range  ballistic  missile  target  with  a  ripple 
launch  of  two  PAC-3  CRI  missiles.  The  first  PAC-3 
missile  intercepted  the  target. 

-  During  PDB-7  flight  test  P7-3  in  November  2011,  Patriot 
engaged  a  short-range  ballistic  missile  target  with  a  ripple 
launch  of  two  GEM-T  missiles.  The  first  GEM-T  missile 
intercepted  the  target. 

-  During  PDB-7  flight  test  P7-2  in  November  2011,  Patriot 
engaged  two  short-range  ballistic  missile  targets  with  two 
ripple  launches  of  GEM-T/GEM-C  missiles.  The  first 
GEM-T  missile  of  each  ripple  engagement  intercepted  its 
target. 

-  During  PDB-7  flight  test  P7-1  in  March  2012,  Patriot  fired 
a  GEM-T  missile  at  one  cruise  missile  target  and  a  GEM-C 
missile  at  a  second  cruise  missile  target.  Both  Patriot 
missiles  intercepted  their  targets. 

•  During  the  first  Integrated  Fire  Control  flight  test  (IFC-1)  at 
the  Utah  Test  and  Training  Range  in  April  2012,  Patriot  fired  a 
PAC-3  CRI  missile  at  a  cruise  missile  target  using  a  Joint  Land 
Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 
cue.  The  PAC-3  missile  intercepted  the  target. 

•  During  the  first  MEADS  flight  test,  the  Launcher/Missile 
Characterization  Test  at  WSMR  in  November  2011,  MEADS 
fired  an  MSE  missile  at  a  simulated  target. 

•  During  the  FMS  G-2  missile  flight  test  at  WSMR  in 
March  2012,  Patriot  intercepted  a  cruise  missile  target  with  a 
GEM-T  missile. 

•  During  the  FMS  P-l/P-2  missile  flight  test  at  WSMR  in 
March  2012,  Patriot  engaged  a  short-range  ballistic  missile 
target  with  a  ripple  launch  of  two  PAC-3  missiles.  The  first 
PAC-3  missile  intercepted  the  target. 

•  During  the  FMS  G-3/G-6/G-7  missile  flight  test  at  WSMR  in 
May  2012,  Patriot  failed  to  intercept  a  short-range  ballistic 
missile  target  during  the  G-3  portion  of  the  mission  because  of 
a  missile  launch  sequence  failure.  As  a  result,  the  G-3  portion 
of  the  mission  was  repeated  using  a  backup  short-range 
ballistic  missile  target,  which  Patriot  intercepted  using  a 
GEM-T  missile.  This  was  followed  by  the  launch  of  another 


short-range  ballistic  missile  target,  which  Patriot  engaged 
using  a  ripple  launch  of  two  GEM-T  missiles.  The  first 
GEM-T  missile  intercepted  the  target. 

•  During  the  FMS  G-4/G-5  missile  flight  test  at  Eglin  AFB, 
Florida,  in  June  2012,  Patriot  performed  near-simultaneous 
intercepts  over  water  of  two  air-breathing  targets  using  GEM-T 
missiles. 

•  During  the  FMS  P-3/P-4  missile  flight  test  at  WSMR  in 
September  2012,  Patriot  engaged  a  short-range  ballistic 
missile  target  with  a  ripple  launch  of  two  PAC-3  CRI  missiles. 
The  first  PAC-3  missile  intercepted  the  target.  This  mission 
concurrently  fulfilled  a  long  standing  PAC-3  Engineering 
Manufacturing  and  Development  phase  requirement. 

•  During  Flight  Test  lntegrated-0 1  (FTI-01)  in  October  2012  at 
the  Reagan  Test  Site,  Patriot  performed  a  near-simultaneous 
engagement  of  a  short-range  ballistic  missile  target  with  two 
PAC-3  interceptors  and  a  cruise  missile  target  with  another 
PAC-3  interceptor.  FTI-01  was  the  first  integrated  flight  test 
with  multiple  firing  elements  (Aegis  Ballistic  Missile  Defense 
[BMD],  Terminal  Fligh-Altitude  Area  Defense  [THAAD],  and 
Patriot)  engaging  multiple  ballistic  missile  and  air-breathing 
targets  in  a  realistic  BMDS-level  architecture.  Patriot 
successfully  intercepted  both  of  its  targets. 

Assessment 

•  The  F1WIL  phase  of  the  PDB-7  LUT  was  to  have  included 
equal  numbers  of  runs  with  and  without  simulated  MSE 
missiles.  All  planned  runs  without  MSE  missiles  were 
completed,  but  Patriot  system  availability  problems  led  to  only 
20  percent  of  the  MSE  runs  being  accomplished.  As  a  result, 
the  effectiveness  assessment  of  the  Patriot  PDB-7  system 
with  MSE  missiles  will  be  limited  until  the  Army  conducts  a 
dedicated  F1WIL  test  with  simulated  MSE  missiles.  Additional 
F1W1L  testing  with  MSE  missiles  is  planned  as  part  of  PDB-7 
regression  testing  scheduled  to  complete  in  November  2012. 

•  Data  analysis  is  ongoing,  but  preliminary  results  indicate 
that  Patriot  training  remains  inadequate  to  prepare  operators 
for  complex  Patriot  engagements.  This  was  true  during 

the  PDB-6.5  and  PDB-6  LUTs  as  well.  This  problem 
was  exacerbated  in  the  PDB-7  LUT  because  many  of 
the  experienced  Patriot  operators  in  the  test  unit  were 
understandably  transferred  to  deploying  units  prior  to  the  LUT, 
resulting  in  many  inexperienced  users  and  a  high  variability  in 
Soldier  proficiency  across  the  test  unit. 

•  Calculation  of  Patriot  system  reliability  such  as  Mean  Time 
Between  Critical  Mission  Failure  using  field  data  is  being 
investigated.  Critical  field  data  including  total  operating  hours 
and  numbers  of  critical  mission  failures  for  each  Patriot  battery 
major  end  item  may  not  be  accurate. 

•  During  the  PDB-7  LUT  operational  missile  flight  test 
(P7L- 1/2/3),  Patriot  demonstrated  the  capability  to  search, 
detect,  track,  engage,  and  intercept  both  a  tactical  ballistic 
missile  target  and  a  cruise  missile  target  with  GEM-T  missiles. 
Patriot  intercepted  the  cruise  missile  target  in  the  debris  field 
caused  by  the  intercept  of  the  first  tactical  ballistic  missile 
target  and  the  self-destruction  of  the  second  tactical  ballistic 
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missile  target.  However,  the  following  problems  were 
observed  during  this  test: 

-  Patriot  was  to  have  engaged  the  first  tactical  ballistic 
missile  target  with  two  GEM-T  missiles,  but  the  launcher 
incorrectly  reported  a  missile  count  of  zero  after  the  first 
GEM-T  missile  launched  so  a  PAC-3  missile  was  launched 
instead. 

-  Patriot  engaged  the  second  tactical  ballistic  missile  target 
with  two  PAC-3  missiles,  but  the  target  broke  up  before  the 
missiles  reached  it.  The  cause  of  this  target  failure  is  under 
investigation. 

-  Patriot  was  to  have  had  two  GEM-T  missiles  available 
to  engage  the  cruise  missile  target,  but  one  could  not  be 
armed.  The  backup  missile  was  not  needed  however,  as  the 
first  GEM-T  missile  launched  successfully  and  intercepted 
the  target. 

-  Although  the  Patriot  crews  were  not  supposed  to  be 
notified  when  the  targets  were  launched,  a  test  conduct 
error  led  to  them  hearing  the  range  communications 
network  during  the  launches.  It  is  unknown  what  effect,  if 
any,  this  had  on  the  test. 

•  The  Patriot  system  met  most  of  the  test  objectives  during  the 
PDB-7  DT&E.  However,  there  were  some  incidents  during 
the  ground  testing  portion  when  Patriot  did  not  properly 
transmit  messages,  had  degraded  track  triangulation  between 
batteries,  was  unable  to  isolate  faults  and  had  to  be  rebooted, 
selected  a  launcher  with  no  available  missiles,  and  selected 
less  preferred  missiles  against  threats  (e.g.,  a  GEM  against  a 
long-range  tactical  ballistic  missile  or  a  PAC-3  missile  against 
a  threat  aircraft).  The  Patriot  system  did  not  meet  its  reliability 
requirements  during  this  test. 

•  During  PDB-7  flight  test  P7-4,  Patriot  demonstrated  the 
capability  to  search,  detect,  track,  engage,  and  intercept  a 
tactical  ballistic  missile  target  with  PAC-3  missiles  in  a  ripple 
engagement.  There  were  some  discrepancies  between  the 
performance  of  the  second  PAC-3  missile  during  its  initial  turn 
and  pre-flight  simulations.  The  causes  of  these  discrepancies 
are  still  under  investigation.  One  of  the  two  non-firing  Patriot 
batteries  did  not  collect  data  during  the  P7-4  flight  test.  The 
affected  non-firing  battery  restarted  the  data  collection  system 
multiple  times  in  an  effort  to  fix  the  problem,  but  it  was  not 
resolved  before  the  engagement. 

•  During  PDB-7  flight  test  P7-3,  Patriot  demonstrated  the 
capability  to  search,  detect,  track,  engage,  and  intercept  a 
tactical  ballistic  missile  target  with  GEM  missiles  in  a  ripple 
engagement. 

•  During  PDB-7  flight  test  P7-2,  Patriot  demonstrated  the 
capability  to  detect,  engage,  and  intercept  short-range  ballistic 
missile  targets  with  GEM-T  missiles.  Patriot  conducted  the 
second  engagement  in  the  presence  of  the  debris  cloud  caused 
by  the  first  intercept. 

•  During  PDB-7  flight  test  P7-1,  Patriot  demonstrated  the 
capability  to  detect,  engage,  and  intercept  cruise  missile  targets 
in  clutter  with  GEM-T  and  GEM-C  missiles. 

•  During  IFC-1,  Patriot  demonstrated  the  capability  to  use  the 
Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted 


Sensor  System  cuing  data  to  engage  a  cruise  missile  with  a 
PAC-3  missile.  The  PAC-3  missile  intercepted  the  target. 

•  During  the  Launcher/Missile  Characterization  Test,  an  MSE 
missile  was  launched  at  a  70-degree  angle,  performed  an 
out-of-plane  maneuver,  and  followed  the  predicted  flight  path 
to  the  simulated  target,  which  was  120  degrees  off  the  launch 
axis.  All  test  objectives  were  met. 

•  Patriot  intercepted  four  of  five  tactical  ballistic  missile  targets 
and  three  of  three  air-breathing  threat/cruise  missile  targets 
during  five  FMS  flight  tests. 

-  The  original  target  in  the  FMS  G-3  missile  flight  test  was  to 
be  engaged  with  only  one  GEM-T  missile,  but  the  GEM-T 
missile  had  a  launch  sequence  failure  and  there  was  not 

an  active  back-up  missile  that  could  launch  in  its  place. 
After  this  failure,  a  back-up  tactical  ballistic  missile  target 
and  back-up  GEM-T  missile  were  activated.  The  back-up 
GEM-T  missile  engaged  and  intercepted  the  back-up 
tactical  ballistic  missile  target. 

-  The  FMS  G-6/G-7  missile  flight  test  was  a  GEM-T  ripple 
engagement  of  another  tactical  ballistic  missile  target  in 
the  debris  caused  by  the  G-3  intercept.  The  first  GEM-T 
had  a  launch  sequence  failure.  However,  there  was  a 
live  back-up  missile  for  this  engagement  so  two  GEM-T 
missiles  were  launched,  the  first  of  which  intercepted  the 
target. 

•  During  FTI-01,  Patriot  demonstrated  the  capability  to  detect, 
track,  engage,  intercept,  and  kill  both  a  tactical  ballistic 
missile  target  and  a  cruise  missile  target  with  PAC-3  missiles. 
There  was  a  Patriot  radar  fault  between  the  cruise  missile  and 
ballistic  missile  engagements,  but  the  system  recovered  and 
was  able  to  conduct  a  nominal  engagement.  The  root  cause 
of  the  radar  fault  is  under  investigation.  All  PAC-3  missile 
subsystems  performed  as  expected.  The  Patriot  engagements 
were  conducted  in  the  debris  field  from  the  THAAD  intercept 
and  Patriot  debris  mitigation  was  nominal.  Aegis  BMD  failed 
to  intercept  its  ballistic  missile  target  during  FTI-01.  The 
Missile  Defense  Agency  did  not  set  up  the  flight  test  so  Patriot 
could  intercept  targets  that  Aegis  or  THAAD  missed,  although 
DOT&E  had  recommended  this  be  a  feature  of  BMDS  flight 
testing  (previous  recommendation  #6  below). 

•  Continuing  obstacles  to  adequate  T&E  of  the  Patriot  system 
include: 

-  Limitations  to  the  lethality  information  available  for 
aircraft,  cruise  missile,  and  air-to-surface  missile  threats 
used  to  assess  end-to-end  system  effectiveness. 

-  The  lack  of  a  robust  interoperability  event  during  PDB-7 
testing. 

-  The  lack  of  a  robust  Force  Development  Experiment, 
preventing  the  Army  from  thoroughly  examining  tactical 
standard  operating  procedures  prior  to  developing  Patriot 
PDB-7  tactics,  techniques,  and  procedures.  As  a  result, 
the  engagement  procedures  used  during  the  PDB-7  LUT 
against  some  threats  may  have  led  to  decreased  system 
performance. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  satisfactorily 

addressed  10  of  the  previous  18  open  recommendations.  The 

Army  should  still  address  the  following  recommendations: 

1.  Conduct  Patriot  testing  during  joint  and  coalition  exercises. 

2.  Upgrade  the  Patriot  HW1L  systems  to  model  electronic 
countermeasures  and  identification  friend-or-foe  systems. 

3.  Conduct  a  Patriot  flight  test  against  an  anti-radiation  missile 
target  to  validate  models  and  simulations. 

4.  Review  the  risks  of  not  conducting  all  flight  tests  against 
ballistic  missile  targets  using  two  interceptors. 

5.  Improve  Patriot  training. 

6.  Have  Patriot  participate  with  live  interceptors  in  THAAD 
flight  testing. 

7.  Conduct  all  operational  testing  regression  tests  with 
representative  Soldier  operators. 

8.  Conduct  a  robust  Force  Development  Experiment  prior  to 
the  PDB-8  IOT&E  to  ensure  that  tactics,  techniques,  and 


procedures  are  adequate  to  support  a  successful  operational 
test. 

FY12  Recommendations.  In  addition  to  addressing  the  above 

recommendations,  the  Army  should: 

1.  Collect  reliability  data  on  Patriot  systems  in  the  field, 
including  total  operating  hours  and  numbers  of  critical 
mission  failures  for  each  Patriot  battery  major  end  item,  so 
that  the  Mean  Time  Between  Critical  Mission  Failure  can 
be  calculated. 

2.  Ensure  that  test  units  for  future  Patriot  operational  tests 
have  operationally  representative  distributions  in  Soldier 
proficiency  by  limiting  transfers  of  experienced  personnel 
to  other  units  before  the  end  of  testing. 

3.  Conduct  future  operational  flight  tests  with  unannounced 
target  launches  within  extended  launch  windows  to  improve 
operational  realism. 
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Precision  Guidance  Kit  (PGK) 


Executive  Summary 

•  In  January  2011,  the  Army  restructured  the  Precision  Guidance 
Kit  (PGK)  Increment  1  program  to  address  any  necessary 
schedule  changes  due  to  reliability  test  failures  that  occurred 
in  August  2010. 

•  Baseline  reliability  testing  in  August  2011  demonstrated  that 
the  contractor  and  program  manager  resolved  the  system’s 
main  reliability  problems  identified  in  2010. 

•  In  March  2012,  the  program  received  approval  to  accelerate 
fielding  to  Operation  Enduring  Freedom  in  FY13.  The 
program  currently  has  two  tracks  -  the  restructured  baseline 
Program  of  Record  and  the  Urgent  Materiel  Release  (UMR). 

•  The  program  is  currently  undergoing  government  qualification 
testing  and  is  revising  the  PGK  Test  and  Evaluation  Master 
Plan  to  support  a  Milestone  C  decision. 

•  In  late  October  2012,  as  part  of  the  UMR  program,  the 
Army  conducted  an  Early  User  Assessment  that  provided 
Soldiers  their  first  opportunity  to  fire  the  PGK.  The  howitzer 
crew  performed  their  tasks  successfully  in  six  operationally 
realistic  end-to-end  missions  firing  a  total  of  20  PGKs  from  an 
M777A2  howitzer.  Preliminary  data  indicate  that  the  PGK’s 
accuracy  is  well  within  requirements. 

•  The  PGK  experienced  two  reliability  failures.  The  program  is 
investigating  potential  modifications  to  the  GPS  antenna  and 
the  ballistic  parameters  used  to  compute  the  firing  mission 
when  firing  an  M549A1  projectile. 

System 

•  The  PGK  is  a  combined  fuze  and  GPS  guidance  kit  that 
improves  the  ballistic  accuracy  of  the  current  stockpile  of  field 
artillery  projectiles. 

•  The  Army  plans  to  develop  PGK  in  two  increments: 

-  Increment  1.  Provide  155  mm  High-Explosive  projectiles 
(M795  and  M549A1)  with  50  meter  Circular  Error 
Probable  (CEP). 

-  Increment  2.  Improve  delivery  accuracy  to  30  meters 
(threshold)  and  20  meters  (objective)  CEP,  as  well  as  add 
anti-jam  capability  as  a  threshold  requirement. 

•  The  PGK  will  operate  with  existing  and  developmental 
artillery  systems  that  have  digital  fire  control  systems  and 


inductive  fuze  setters  such  as  the  M777A2  Lightweight  Towed 
Howitzer  and  the  M109A6  Paladin  Self-Propelled  Howitzer. 

Mission 

Field  Artillery  units  will  use  PGK  to  provide  indirect  fire  support 
with  30  -  50  meters  accuracy  to  combat  maneuver  units  in  all 
types  of  weather  and  terrain.  Artillery  units  will  use  PGK  to 
achieve  comparable  effects  of  conventionally  fuzed  projectiles 
using  fewer  rounds  and  thus  reducing  collateral  damage. 

Major  Contractor 

Alliant-Techsystems  Advanced  Weapons  Division  -  Plymouth, 
Minnesota 


Activity 

•  In  January  2011,  the  Army  restructured  the  PGK  Increment  1 
program  to  address  schedule  changes  due  to  the  reliability 
test  failures  that  occurred  in  August  2010.  The  restructured 
program  features  a  Milestone  C  decision  in  2QFY13,  an  Initial 
Operational  Test  (IOT)  in  1QFY14,  a  Full-Rate  Production 
decision  in  2QFY 14,  and  an  Initial  Operational  Capability  in 
3QFY14. 


•  The  PGK  Increment  1  has  three  Milestone  C  entrance 
criteria:  interoperability,  reliability,  and  accuracy.  The 
program  successfully  demonstrated  interoperability  during 
DOT&E-approved  testing  in  2009. 

•  In  August  2011,  the  Army  tested  a  modified  PGK  to  address 
known  failure  modes  in  order  to  demonstrate  satisfactory 
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baseline  reliability  to  support  entrance  into  the  next  phase  of 
Engineering  and  Manufacturing  Development. 

•  In  March  2012,  the  program  received  approval  to  accelerate 
fielding  to  Operation  Enduring  Freedom  in  FY13.  Thus,  the 
program  currently  has  two  tracks  -  the  baseline  Program  of 
Record  and  the  UMR. 

•  The  program  is  currently  undergoing  government  qualification 
testing,  which  includes  Sequential  Environmental  Tests  for 
safety  and  performance. 

•  The  program  is  revising  the  PGK  Test  and  Evaluation  Master 
Plan  to  support  a  Milestone  C  decision  in  2QFY13.  Following 
Milestone  C,  the  program  intends  to  begin  the  manufacturing 
of  fuzes  for  the  IOT  and  the  validation  of  the  production  line 
to  support  the  Full-Rate  Production  decision.  The  program 
plans  to  conduct  an  IOT  in  1QFY14  at  Yuma  Proving  Ground, 
Arizona. 

•  In  late  October  2012,  as  part  of  the  UMR  program,  the 
Army  conducted  an  Early  User  Assessment  that  provided 
Soldiers  their  first  opportunity  to  fire  the  PGK.  The  howitzer 
crew  performed  their  tasks  successfully  in  six  operationally 
realistic  end-to-end  missions  firing  a  total  of  20  PGKs  from  an 
M777A2  howitzer.  These  firings  provided  both  accuracy  and 
reliability  as  input  to  the  UMR  decision  planned  for  2QFY13. 

Assessment 

•  In  August  2011,  the  Baseline  Reliability  testing  of  48  fuzes 
to  support  entrance  into  the  next  phase  of  Engineering 
and  Manufacturing  Development  demonstrated  84  percent 
reliability  with  80  percent  confidence,  exceeding  the  required 
83  percent  with  80  percent  confidence.  This  was  the  target 
reliability  for  a  successful  program  and  approval  of  an 
accelerated  fielding  effort. 

•  This  reliability  estimate  is  on  the  reliability  growth  curve  for 
the  program  leading  to  the  Capability  Production  Document 
requirement  of  92  percent  by  Initial  Operational  Capability. 
However,  in  the  July  2012  Sequential  Environmental 

Test  -  Safety,  three  rounds  fell  significantly  short  of  the  target. 
This  indicates  a  possible  new  failure  mode.  These  failures 


are  currently  undergoing  failure  analysis.  The  Army  will 
determine  the  effect  of  these  failures  on  the  program  schedule 
upon  completion  of  the  failure  analysis. 

•  The  program  is  meeting  accuracy  requirements  for  the 
M795  and  the  M549A1  155  mm  high-explosive  projectiles 
at  low-  and  mid-firing  angles.  The  program  has  focused 
considerable  resources  and  is  making  some  progress  in 
enhancing  the  PGK  accuracy  at  higher  firing  angles. 

•  Results  from  the  operational  Early  User  Assessment  of 

20  fuzes  in  October  2012  indicate  that  the  PGK  demonstrated 
an  85  percent  reliability  with  a  lower  80  percent  confidence 
bound  of  74  percent.  Two  of  the  three  failures  in  the  small 
sample  replicated  a  known  failure  mode  with  a  planned 
corrective  action  in  the  Program  of  Record  track  but  not  in 
the  initial  UMR  lots.  Preliminary  accuracy  data  indicate  that 
overall,  the  PGK’s  accuracy  met  the  threshold  requirement 
of  50  meters  CEP,  demonstrating  a  32-meter  CEP  although 
there  was  considerable  variability  in  the  errors  -  ranging 
from  9  meters  to  131  meters.  The  program  is  investigating 
modifications  to  the  GPS  antenna  and  the  ballistic  parameters 
used  to  compute  the  firing  mission  when  firing  an  M549A1 
projectile.  A  change  to  the  digital  fire  control  software  is  in 
progress,  which  should  be  implemented  with  the  upcoming 
UMR  fielding. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  has 
satisfactorily  addressed  all  previous  recommendations. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Continue  to  closely  monitor  developmental  testing  to 
ensure  that  the  identified  corrective  actions  for  the  known 
deficiencies  are  satisfactory  and  do  not  adversely  affect 
other  performance  parameters  prior  to  Milestone  C. 

2.  Determine  the  root  cause  of  the  rounds  that  fell  short  of  the 
target  and  implement  corrective  action  prior  to  Milestone  C. 

3.  Determine  the  causes  of  the  substantial  variance  in  accuracy 
observed  in  the  Early  User  Assessment. 
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Q-53  Counterfire  Target  Acquisition  Radar  System 


Executive  Summary 

•  In  February  2012,  the  Army  selected  Lockheed  Martin  as 
the  primary  contractor  for  the  Q-53  Program  of  Record.  The 
Army  plans  to  buy  136  Q-53  radars  as  part  of  the  Program  of 
Record. 

•  The  Army  contracted  with  Lockheed  Martin  to  build  38 
Quick  Reaction  Capability  (QRC)  radars  to  support  an  Urgent 
Materiel  Release.  The  QRC  production  buy  was  completed  in 
March  2012.  Five  QRC  systems  are  operating  in  Afghanistan. 

•  The  Army  completed  the  first  reliability  test  of  the  Program 
of  Record  radar.  The  radar’s  system  abort  rate  was  better 
than  the  rate  observed  in  the  system  demonstration  prior 

to  the  February  2012  Milestone  C  update.  However,  the 
demonstrated  reliability  rate  is  below  the  predicted  rates 
needed  to  reach  reliability  requirements  by  the  IOT&E. 

System 

•  The  Army  changed  the  designation  of  the  Enhanced 
AN/TPQ-36  (EQ-36)  radar  to  the  AN/TPQ-53  (Q-53)  radar  in 
September  2011. 

•  The  Q-53  is  a  mobile  radar  system  designed  to  detect,  classify, 
and  track  projectiles  fired  from  mortar,  artillery,  and  rocket 
systems  using  a  90-degree  or  continuous  360-degree  sector 
search. 

•  The  radar  provides  target  location  of  threat  indirect  fire 
systems  with  sufficient  accuracy  for  effective  counterfire. 

•  The  Q-53  is  designed  to  operate  with  the  Counter  -  Rocket, 
Artillery,  Missile  (CRAM)  system  and  the  future  Indirect  Fire 
Protection  Capability  System. 

•  The  Army  intends  to  field  the  Q-53  radar  to  the  sensor 
platoons  in  Brigade  Combat  Teams  and  Fire  Brigades  to 
replace  the  legacy  AN/TPQ-36  and  AN/TPQ-37  Firefinder 
Radars. 

•  The  Q-53  is  operated  by  a  crew  of  four  Soldiers  and 
transportable  by  C-17  aircraft,  with  battlefield  mobility 
provided  by  two  Family  of  Medium  Tactical  Vehicle  trucks. 


•  The  Army  contracted  with  Lockheed  Martin  Missile  Systems 
and  Sensors  to  develop  and  field  38  QRC  radars  to  support  an 
Urgent  Material  Release.  Fielding  began  in  2010  with  five 
systems  operating  in  Afghanistan. 

•  The  Army  intends  to  produce  136  Program  of  Record  Q-53 
radars. 

Mission 

Field  Artillery  units  protect  friendly  forces  by  employing  the 
Q-53  radar  to  determine  timely  and  accurate  location  of  threat 
rocket,  artillery,  and  mortar  systems  for  defeat  with  counterfire 
engagements.  Air  Defense  Artillery  units  will  use  the  Q-53  radar 
integrated  into  the  CRAM  and  Indirect  Fire  Protection  Capability 
System  to  warn  friendly  forces  and  to  engage  incoming  threat 
indirect  fires. 

Major  Contractor 

Lockheed  Martin  Missile  Systems  and  Sensors  -  Syracuse,  New 
York 


Activity 

•  The  Army  conducted  a  Milestone  C  update  on 

February  27,  2012.  The  Army  selected  Lockheed  Martin  as 
the  primary  contractor. 

•  The  Army  purchased  33  systems  as  part  of  the  Milestone  C 
update  decision. 

•  The  Army  intends  to  purchase  an  additional  18  systems  after 
the  Limited  User  Test  (LUT)  scheduled  for  October  22  through 
November  8,  2012,  and  the  remaining  85  systems  at  the 
4QFY14  Full-Rate  Production  decision  following  the  IOT&E 
in  1QFY14. 


•  The  Army  completed  the  first  Program  of  Record  reliability 
and  performance  tests  at  Yuma  Proving  Ground,  Arizona,  from 
May  15  through  August  17,  2012.  The  contractor-operated 
radars  completed  1,662  test  hours.  The  two  radars  operated 
continuously  in  72-hour  cycles  and  made  moves  representative 
of  the  distances  and  terrain  expected  in  an  operational 
environment. 

•  Environmental  chamber  testing  began  at  White  Sands  Missile 
Range,  New  Mexico,  in  August  2012.  It  will  be  completed  by 
the  end  of  2012. 
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Assessment 

•  Based  on  developmental  testing  at  Yuma  Proving  Ground, 
the  radar’s  reliability  has  improved  since  the  system 
demonstration,  but  is  below  the  growth  projections  needed  to 
reach  the  reliability  requirement  at  IOT&E. 

•  During  the  system  demonstration  in  2011,  the  radar  averaged 
1  system  abort  every  30  hours.  During  2012  developmental 
testing,  the  Program  Office  made  three  significant 
configuration  changes  to  address  system  aborts. 

-  During  the  first  2  configuration  changes,  the  radars  averaged 
1  system  abort  every  103  hours  and  1  system  abort  every 
238  hours,  respectively. 

-  During  limited  testing  of  the  final  configuration  (298  test 
hours),  the  radars  averaged  a  system  abort  every  75  hours. 

•  To  reach  the  reliability  threshold,  the  Program  Office  expected 
the  radar  to  average  1  system  abort  every  257  hours.  The 
Army  determined  two  of  the  four  system  aborts  during  testing 
of  the  final  configuration  were  due  to  problems  with  the 


user  manual  and  training  deficiencies,  unrelated  to  the  final 
configuration.  The  user  manual  and  training  will  be  updated 
prior  to  the  LUT.  The  remaining  two  system  aborts  were 
software  related  and  will  not  be  addressed  prior  to  the  LUT. 

•  The  radar  met  performance  requirements  during  limited 
developmental  testing  in  FY12.  More  extensive  performance 
testing  is  planned  for  the  LUT  in  October  and  November  2012. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  satisfactorily 
addressed  all  of  the  FY11  recommendations. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Conduct  future  developmental  reliability  tests  with  trained 
civilian  crews  and  limited  contractor  involvement. 

2.  Continue  dedicated  reliability  testing  focusing  on  system 
aborts. 
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Spider  XM-7  Network  Command  Munition 


Executive  Summary 

•  The  Army  uses  Spider  instead  of  persistent  landmines  to 
comply  with  the  requirements  of  the  2004  National  Landmine 
Policy. 

•  The  Army  fielded  Spider  Low-Rate  Initial  Production  (LRIP) 
systems  to  deployed  and  non-deployed  units  during  2012. 

•  At  the  request  of  the  Army,  DOT&E  published  a  Beyond 
Low-Rate  Initial  Production  (BLR1P)  report  in  February  2012 
to  support  a  Full-Rate  Production  (FRP)  decision  in  2012. 
Following  publication  of  this  report,  the  Army  postponed  the 
Spider  FRP  decision  until  3QFY13. 

•  The  Army  continued  corrective  actions  to  address  Spider 
deficiencies  with  system  reliability,  complexity,  and  training 
reported  in  the  February  2012  DOT&E  Spider  BLRIP  Report. 

•  DOT&E  will  report  on  the  operational  effectiveness, 
suitability,  and  survivability  of  the  Spider  system  early  in  2013 
following  a  third  FOT&E.  Based  on  analysis  conducted  to 
date,  Spider  has  demonstrated  effectiveness  and  lethality  with 
poorly  demonstrated  suitability. 

System 

•  The  Army  intends  to  use  Spider  as  a  landmine  alternative  to 
satisfy  the  anti-personnel  munition  requirements  outlined  in 
the  2004  National  Landmine  Policy  that  directs  the  DoD  to: 

-  End  use  of  persistent  landmines  after  2010 

-  Incorporate  self-destructing  and  self-deactivating 
technologies  in  alternatives  to  current  persistent  landmines 

•  A  Spider  munition  field  includes: 

-  Up  to  63  Munition  Control  Units,  each  housing  up  to  6 
miniature  grenade  launchers  or  munition  adapter  modules 
for  remote  electrical  and  non-electrical  firing  capabilities 

-  A  remote  control  station,  used  by  the  operator  to  maintain 
“man-in-the-loop”  control  of  all  munitions  in  a  field 

-  A  communications  relay  device  known  as  a  “repeater”  for 
use  in  difficult  terrain  or  at  extended  ranges 

•  Spider  incorporates  self-destructing  and  self-deactivating 
technologies  to  reduce  residual  risks  to  non-combatants. 


Mission 

Maneuver  or  engineer  units  employ  Spider  to  establish  a  force 
protection  obstacle  or  as  a  stand-alone  force  protection  system 
in  all  environments  and  in  all  terrains  in  order  to  accomplish  the 
following  missions: 

•  Protect  the  Force 

•  Shape  the  Battlefield 

•  Provide  Early  Warning 

•  Delay  and  Attrite  Enemy  Forces 

Major  Contractors 

•  Command  and  Control  hardware  and  software:  Textron 
Defense  Systems  -  Wilmington,  Massachusetts 

•  Munition  Control  Unit  and  Miniature  Grenade 
Launcher:  Alliant-Techsystems,  Advanced  Weapons 
Division  -  Plymouth,  Minnesota 


Activity 

•  The  Army  continued  fielding  Spider  LRIP  systems  to  deployed 
and  non-deployed  units.  Flome  Station,  Combat  Training 
Center,  and  in-theater  training  were  provided  by  the  materiel 
and  combat  developers  as  part  of  the  fielding  package.  In 
January  2012,  the  Army  Milestone  Decision  Authority 
approved  the  production  of  additional  LRIP  systems  to  support 
continued  fielding  prior  to  an  FRP  decision. 

•  Based  on  demonstrated  performance  in  the  May  2010 
Spider  FOT&E  and  the  June  2011  Spider  Limited  User  Test, 


the  Army  requested  DOT&E  publish  a  BLRIP  report  in 
February  2012.  Following  publication  of  the  report,  the  Army 
postponed  the  Spider  FRP  decision  until  3QFY13. 

•  The  Army  continued  corrective  actions  to  address  Spider 
deficiencies  with  system  reliability,  complexity,  and  training 
reported  in  the  February  2012  DOT&E  Spider  BLRIP  Report. 

•  The  Army  and  DOT&E  finalized  planning  for  the  third 
FOT&E  in  October  2012  to  demonstrate  corrective  actions  in 
an  operationally  realistic  environment. 
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•  In  October  2012,  DOT&E  approved  an  updated  Test  and 
Evaluation  Master  Plan.  The  update  addressed  the  scope  and 
execution  of  a  comprehensive  FOT&E  to  demonstrate  Spider 
operational  effectiveness  and  suitability  in  support  of  an  Army 
FRP  decision. 

Assessment 

•  The  Army  conducted  the  Spider  XM7  Limited  User  Test  in 
accordance  with  the  DOT&E-approved  test  plan  in  June  2011 
at  White  Sands  Missile  Range,  New  Mexico. 

•  DOT&E  intends  to  publish  an  updated  evaluation  on  Spider 
XM7  early  in  FY13  based  on  the  October  2012  FOT&E. 

•  DOT&E  made  the  following  assessment  based  on  the  BLRIP 
report  in  February  2012: 

-  Spider  provides  the  following  enhanced  capabilities  not 

previously  available  with  anti-personnel  land  munition 

systems: 

■  “Man-in-the-loop”  positive  control  of  both  lethal  and 
non-lethal  munitions 

■  Remote  electrical  and  non-electrical  firing  capabilities  for 
munitions  and  demolitions  to  a  range  of  4  kilometers 

■  Capability  to  fire  a  single  munition  or  multiple  munitions 
simultaneously 

■  Capability  to  collect  situational  awareness  information 
through  tripline  activation  by  threat  personnel 

-  Spider  has  demonstrated  effectiveness  and  lethality. 

■  A  properly  trained  unit  can  successfully  emplace  and 
operate  a  Spider  munition  field  in  order  to  provide 
doctrinal  protective  obstacle  effects  -  warn  of  threat 
activity  and  mitigate  or  prevent  threat  actions. 


■  Units  employing  Spider  can  utilize  both  non-lethal  and 
lethal  munitions  to  achieve  desired  force  escalation 
capabilities. 

-  Spider  has  demonstrated  poor  suitability. 

■  Spider  is  more  complex  than  its  predecessor  system 
and  requires  Soldiers  to  receive  extensive  initial  and 
sustainment  training  to  maintain  proficiency. 

■  The  Spider  system’s  Munition  Control  Unit  has  not 
demonstrated  the  required  reliability  in  a  comprehensive 
operationally  realistic  environment. 

■  Extensive  battery  management  requirements  and 
increased  unit  transportation  requirements  create  a 
logistics  planning  challenge  for  units  employing  Spider. 

•  The  Spider  program  demonstrated  in  contractor  and 
government  testing  corrective  actions  to  address  reliability, 
complexity,  and  training  deficiencies  reported  in  the 
February  2012  DOT&E  Spider  BLRIP  Report.  These 
corrective  actions  are  ready  for  further  testing  in  an 
operationally  realistic  environment. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  initiated 
actions  to  address  the  previous  recommendations. 

•  FY12  Recommendation. 

1 .  The  Army  should  closely  monitor  the  results  of  the 
October  2012  FOT&E  and  be  prepared  to  address 
shortcomings  and  deficiencies  as  necessary  to  support  a 
3QFY13  FRP  decision. 
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Stryker  Mobile  Gun  System  (MGS) 


Executive  Summary 

•  During  the  December  2010  Stryker  Double-V  Hull  (DVH) 
Configuration  Steering  Board,  the  Army  decided  not  to  pursue 
full-rate  production  for  the  flat-bottom  Stryker  Mobile  Gun 
System  (MGS). 

•  Developmental  tests  have  failed  to  replicate  the  coaxial 
machine  gun  deficiency  in  which  brass  and  links  falling 
into  the  ammunition  storage  box  cause  the  low  ammunition 
sensor  to  fail  to  activate.  This  deficiency  was  noted  in  the 
August  2011  Engineering  Change  Order  Block  III  validation 
gunnery  and  has  been  noted  by  MGS  crews  returning  from 
Afghanistan  and  during  their  training  gunneries. 

•  Live  fire  testing  indicates  performance  deficiencies  in  the 
protection  provided  by  the  Stryker  Reactive  Armor  Tiles 
(SRAT)  II  configuration  for  MGS.  The  details  are  classified. 

System 

•  The  Stryker  Family  of  Vehicles  consists  of  two  variants  on  a 
common  vehicle  platform:  Infantry  Carrier  Vehicle  and  the 
MGS.  There  are  eight  configurations  of  the  Infantry  Carrier 
Vehicle  variant. 

•  The  MGS  mission  equipment  includes  the  following: 

-  The  system  integrates  the  Driver’s  Vision  Enhancer 
and  Command,  Control,  Communications,  Computers, 
Intelligence,  Surveillance,  and  Reconnaissance 
components  as  government-furnished  equipment. 

-  The  MGS  provides  the  three-man  crew  with  varying  levels 
of  protection  against  small-arms,  fragmenting  artillery, 
mines,  and  rocket-propelled  grenades  (RPGs).  Add-on  slat 
armor  (high  hard  steel  arranged  in  a  spaced  array)  provides 
RPG  protection. 

•  The  MGS  mission  equipment  includes  the  following: 

-  M68A2  105  mm  cannon  system  with  an  ammunition 
handling  system 

-  Coaxial  7.62  mm  machine  gun  and  a  secondary  M2HB, 
.50-caliber  machine  gun 


-  Full  solution  fire  control  system  with  two-axis  stabilization 

-  Low-profile  turret  meant  to  provide  survivability  against 
specified  threat  munitions 

Mission 

•  The  Stryker  Brigade  Combat  Team  uses  the  MGS  to  create 
openings  in  walls,  destroy  bunkers  and  machine  gun  nests,  and 
defeat  sniper  positions  and  light  armor  threats.  The  primary 
weapon  systems  are  designed  to  be  effective  against  a  range  of 
threats  up  to  T-62  tanks. 

•  The  MGS  operates  as  a  three-vehicle  platoon  organic  to  the 
Stryker  infantry  company  or  as  a  single  vehicle  in  support  of  a 
Stryker  infantry  platoon. 

Major  Contractor 

General  Dynamics  Land  Systems  -  Sterling  Heights,  Michigan 


Activity 

•  During  the  December  2010  Stryker  DVH  Configuration 
Steering  Board,  the  Army  decided  not  to  pursue  full-rate 
production  for  the  Stryker  flat-bottom  MGS.  The  Army 
determined  it  could  not  integrate  the  DVH  design  onto  the 
MGS  platform  without  the  Stryker  Modernization  program  to 
resolve  weight  and  power  shortfalls. 

•  A  total  of  142  MGSs  have  been  produced  and  fielded.  Three 
MGSs  have  been  total  losses  as  a  result  of  battle  damage  so 
the  current  fleet  has  139. 


•  Testing  in  the  August  2011  Engineering  Change  Order 
Block  III  validation  gunnery  confirmed  the  low  ammunition 
sensor  deficiency  as  part  of  the  larger  deficiency,  inadequate 
ready  load  for  the  7.62  coaxial  machine  gun.  This  is  1  of 
the  5  remaining  deficiencies  of  23  identified  in  the  luly  2008 
Acquisition  Decision  Memorandum. 

•  During  2012,  the  contractor  was  not  able  to  replicate  the 
coaxial  machine  gun  low  ammunition  sensor  deficiency  in 
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which  brass  and  links  falling  into  the  ammunition  storage  box 
inadvertently  triggered  the  low  ammunition  sensor. 

•  The  Army  executed  multiple  components  of  the  LFT&E 
program  for  the  MGS  with  SRAT  II,  including  tests  to 
characterize  the  protection  provided  by  the  add-on  armor  and 
full-up  system-level  testing  to  characterize  SRAT  II  integration 
effects  on  the  MGS  mission  equipment. 

•  The  Army,  in  coordination  with  DOT&E,  submitted  the 
seventh  report  to  Congress  in  December  2011,  updating  the 
status  of  actions  taken  by  the  Army  to  correct  or  mitigate  all 
Stryker  MGS  deficiencies,  as  directed  in  Section  115  of  the 
FY09  Duncan  Hunter  National  Defense  Authorization  Act. 

The  Army  will  issue  their  next  report  when  the  remaining  five 
deficiencies  are  corrected. 

Assessment 

•  The  Army  has  yet  to  correct  the  coaxial  machine  gun  low 
ammunition  sensor  deficiency  via  material  fix  or  crew 
workaround.  The  program  manager  has  designed  a  chute 
modification  with  follow-on  testing  to  take  place  in  December. 

•  Live  fire  testing  indicates  performance  deficiencies  in  the 
protection  provided  by  the  SRAT  II  configuration  for  MGS. 
The  details  are  classified. 

•  In  the  2007  Beyond  Low-Rate  Initial  Production  report, 
DOT&E  assessed  the  MGS  as  not  operationally  effective 
when  operating  in  a  degraded  capacity.  DOT&E  assessed 
that  the  gun  pod  can  be  easily  disabled,  causing  the  MGS  to 
operate  in  a  degraded  capacity,  thereby  making  the  MGS  not 


operationally  effective.  Lack  of  adequate  gun  pod  protection 
makes  the  MGS  vulnerable  to  widely  proliferated  threats 
including  RPGs,  which  increases  the  likelihood  of  the  MGS 
operating  in  a  degraded  capacity.  The  Army  has  no  plans  to 
improve  gun  pod  protection. 

•  The  C-130  Transportability  Key  Performance  Parameter  is 
a  design  constraint  that  limits  MGS  capabilities.  Because 
of  size  and  weight  constraints  for  transporting  equipment  on 
the  C-130,  there  is  a  limitation  on  the  size  and  weight  of  the 
MGS.  This  limit  results  in  several  survivability  deficiencies, 
including  protection  of  the  Commander’s  Weapon  Station, 
protection  of  105  mm  ammunition,  gun  pod  protection,  and 
hydraulic  circuit  separation.  If  this  program  moves  forward, 
a  Stryker  modernization  program  will  have  the  opportunity  to 
address  these  deficiencies. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 
the  one  new  recommendation  from  F Y 1 1 . 

•  FY12  Recommendations.  As  part  of  our  coordination  with 
the  Army,  as  directed  in  Section  115  of  the  FY09  National 
Defense  Authorization  Act,  DOT&E  recommended  that  the 
Army: 

1 .  Increase  gun  pod  protection. 

2.  Provide  a  close  out  report  to  Congress  when  the  RPG 
protection  deficiency  and  the  coaxial  low  ammunition 
sensor  deficiency  is  corrected. 
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Stryker  Double-V  Hull  (DVH) 


Executive  Summary 

•  The  Army  rapidly  developed,  tested,  and  fielded  the  Stryker 
Double-V  Hull  (DVH)  in  response  to  needs  from  commanders 
in  Operation  Enduring  Freedom  (OEF)  regarding  Stryker  force 
protection/ survivability  shortfalls  against  underbody  IEDs  and 
blast  threats.  Testing  and  analysis  confirm  that  the  Stryker 
DVH  configurations  improve  Stryker  vehicle  protection 
against  IEDs;  the  details  are  classified. 

•  All  configurations  of  the  Stryker  DVH  are  operationally 
effective  for  deployment  into  Afghanistan. 

•  All  configurations  of  the  Stryker  DVH  are  operationally 
suitable. 

•  DOT&E  published  six  classified  reports  on  FY12  activity 
for  the  following  variants:  the  Stryker  DVH  Commander’s 
Vehicle  (CVV),  Infantry  Carrier  Vehicle  -  Scout  (ICVV-S), 
Mortar  Carrier  Vehicle  (MCVV),  Medical  Evacuation  Vehicle 
(MEW),  Engineer  Squad  Vehicle  (ESVV),  and  Anti-Tank 
Guided  Missile  Vehicle  (ATVV). 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  continues 
to  develop  the  System  Evaluation  Plan  in  support  of  testing  for 
Stryker  DVH  worldwide  fielding. 

System 

•  The  Stryker  DVH  Infantry  Carrier  Vehicle  (ICVV)  is  the  base 
variant  for  seven  additional  DVH  configurations: 

-  Anti-Tank  Guided  Missile  Vehicle  (ATVV) 

-  Commander’s  Vehicle  (CVV) 

-  Engineer  Squad  Vehicle  (ESVV) 

-  Fire  Support  Vehicle  (FSVV) 

-  Mortar  Carrier  Vehicle  (MCVV) 

-  Medical  Evacuation  Vehicle  (MEW) 

-  ICW-Scout  (ICVV-S) 

•  The  ICVV-S  is  a  new  configuration  to  permit  internal  stowage 
of  the  Long-Range  Advance  Scout  Surveillance  System. 

•  The  DVH  configuration  consists  of  a  redesigned  lower  hull, 
energy  attenuating  seats,  and  an  up-armored  driver  station.  An 
upgraded  suspension  and  driveline  are  incorporated  because  of 
the  additional  weight. 

•  At  this  time,  the  Army  does  not  plan  to  purchase  Stryker  DVH 
versions  of  the  Stryker  Reconnaissance  Vehicle;  Mobile  Gun 
System;  or  the  Nuclear,  Biological,  Chemical  Reconnaissance 
Vehicle. 

Mission 

•  Combatant  Commanders  employ  a  DVH-equipped  Stryker 
Brigade  Combat  Team  (SBCT)  as  a  decisive  action  combat 
force  that  conducts  operations  (offensive,  defensive,  stability, 
and  defense  support  of  civil  authorities)  against  conventional 
or  unconventional  enemy  forces  in  all  types  of  terrain  and 
climate  conditions.  In  addition,  it  operates  across  the  range 


XM1256  Infantry  Carrier  Vehicle  XM1257  Engineer  Squad  Vehicle 


XM1252  Mortar  Carrier  Vehicle  XM1253  Medical  Evacuation  Vehicle 


of  military  operations  (major  operations  and  campaigns, 
crisis  response  and  limited  contingency  operations,  military 
engagement,  security  cooperation,  and  deterrence). 

•  The  DVH-equipped  SBCT  has  the  same  mission  profile  as 
a  non  DVH-equipped  SBCT.  The  Army  intends  to  use  the 
DVH  as  Theater  Provided  Equipment  in  Afghanistan,  and 
provide  the  Army  with  a  long-term  worldwide  capability 
to  simultaneously  deploy  SBCTs  into  a  non-permissive 
environment. 

•  The  Army  intends  for  the  Stryker  DVH  to  provide  improved 
survivability  against  IEDs  and  blast  threats,  beyond  the 
protection  provided  by  current  flat-bottom  Stryker  vehicles 
with  OEF  kits. 

Major  Contractor 

General  Dynamics  Land  Systems  -  Sterling  Heights,  Michigan 
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Activity 

•  The  Army  executed  all  live  fire  and  operational  testing  in 
accordance  with  DOT&E-approved  test  plans. 

•  The  Army  executed  multiple  series  of  full-up  system-level  live 
fire  events  against  the  following  DVH  versions:  ESVV,  CVV, 
MCVV,  MEW,  ATW,  and  ICW-S.  The  purpose  of  the 
follow-on  DVH  LFT&E  program  was  to  compare  each  DVH 
configuration’s  1ED  protection  to  existing  OEF-kitted  Stryker 
vehicles,  as  well  as  to  identify  any  configuration-unique 
vulnerabilities  to  underbody  IED  and  blast  threats. 

•  ATEC  completed  developmental,  operational,  and  live  fire 
testing  of  the  following  Stryker  DVH  variants:  ICVV-S, 

CVV,  FSVV,  MCVV,  MEW,  and  ATW  through  3QFY12 
to  characterize  any  degradation  to  reliability,  availability, 
maintainability,  and  cross-country  mobility,  and  compare  DVH 
performance  to  the  Strykers  currently  used  in  OEF.  ATEC 
conducted  operational  testing  of  the  MCVV  at  Yuma  Proving 
Ground,  Arizona,  and  integrated  testing  of  the  MEW  at 
Aberdeen  Proving  Ground,  Maryland. 

•  DOT&E  published  classified  reports  on  the  CVV,  ICVV-S, 
MCVV,  MEW,  ESVV,  and  ATW  variants  on  FY12  activity. 

•  The  Army  corrected  suitability  shortfalls  with  the  driver’s 
station  identified  during  the  initial  ICW  operational  test 
through  an  initiative  called  the  Driver’s  Station  Enhancement 
II  (DSE  II).  The  Army  executed  suitability  and  survivability 
testing  of  the  DSE  II  March  to  July  2012  at  Fort  Lewis, 
Washington;  Aberdeen  Proving  Ground,  Maryland;  and  Yuma 
Proving  Ground,  Arizona. 

•  ATEC  continues  to  develop  the  System  Evaluation  Plan  to 
support  testing  measures  for  worldwide  use  of  Stryker  DVH. 


Assessment 

•  Stryker  DVH  systems  were  rapidly  developed,  tested,  and 
fielded  in  response  to  needs  from  commanders  in  OEF. 
Testing  and  analysis  confirm  that  DVH  systems  improve 
Stryker  vehicle  protection  against  IEDs.  The  details  can  be 
found  in  the  classified  DOT&E  LFT&E  report  on  the  CVV 
(January  2012)  and  Operational  Assessment/LFT&E  reports 
for  ICVV-S  (January  2012),  MCVV  (May  2012),  and  MEW 
(July  2012). 

•  All  configurations  of  the  Stryker  DVH  are  operationally 
effective  for  deployment  into  Afghanistan.  There  were  no 
significant  differences  between  the  Strykers  currently  used  in 
OEF  and  DVH  Strykers  regarding  mobility  and  the  ability  of 
units  equipped  with  the  two  types  of  vehicles  to  accomplish 
the  mission. 

•  All  configurations  of  the  Stryker  DVH  are  operationally 
suitable  to  include  the  driver’s  station.  The  Stryker  DVH 
demonstrated  better  reliability  and  maintainability  than  the 
OEF  variant. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 
the  previous  recommendations. 

•  FY12  Recommendations.  None. 


122 


Stryker  DVH 


ARMY  PROGRAMS 


Warfighter  Information  Network  -  Tactical  (WIN-T) 


Executive  Summary 

•  In  20 1 1 ,  the  Army  conducted  a  Warfighter  Information 
Network  -  Tactical  (WIN-T)  Increment  2  developmental 
test  that  assessed  a  network  of  more  than  70  WIN-T 
Increment  2  communications  nodes.  In  benign  developmental 
test  conditions,  WIN-T  Increment  2  met  its  performance 
requirements,  but  did  not  meet  its  Army-defined  reliability 
requirements. 

•  In  May  2012,  the  Army  conducted  a  WIN-T  Increment  2 
IOT&E  as  a  part  of  the  Network  Integration  Evaluation 
(NIE)  12.2.  The  Soldier  Network  Extension  (SNE),  Tactical 
Relay-Tower  (TR-T),  and  High-Band  Networking  Waveform 
(HNW)  were  not  effective.  All  other  configuration  items  and 
the  Net-Centric  Waveform  (NCW)  were  effective. 

•  DOT&E  provided  details  of  the  IOT&E  test  results  in  the 
WIN-T  Increment  2  Beyond  Low-Rate  Initial  Production 
(BLRIP)  Report  issued  on  September  25,  2012. 

•  WIN-T  Increment  2  is  not  suitable  due  to  poor  reliability  and 
maintainability  and  not  survivable  due  to  deficiencies  noted  in 
the  classified  annex  to  the  DOT&E  BLRIP  report. 

•  In  September  2012,  the  Defense  Acquisition  Executive  (DAE) 
authorized  the  Army  to  procure  an  additional  538  WIN-T 
Increment  2  communication  nodes.  The  DAE  directed  the 
Army  to  conduct  an  FOT&E  to  demonstrate  improvement  of 
the  SNE  and  HNW,  and  to  provide  evidence  that  each  WIN-T 
Increment  2  configuration  item  is  on  track  to  meet  reliability 
and  maintainability  requirements.  The  DAE  directed  the 
Army  to  provide  an  updated  growth  plan  with  growth  curves 
to  achieve  reliability  and  maintainability  requirements  for  each 
WIN-T  Increment  2  configuration  item. 

•  The  program  plans  to  conduct  an  FOT&E  on  the  system 
during  NIE  13.2  in  May  2013. 

System 

•  The  Army  designed  the  WIN-T  as  a  three-tiered 
communications  architecture  (space,  terrestrial,  and  airborne) 
to  serve  as  the  Army’s  high-speed  and  high-capacity  tactical 
communications  network. 

•  The  Army  intends  WIN-T  to  provide  reliable,  secure,  and 
seamless  communications  for  units  operating  at  theater  level 
and  below. 

•  The  WIN-T  program  consists  of  four  increments. 

-  Increment  1:  “Networking  At-the-Halt”  enables  the 
exchange  of  voice,  video,  data,  and  imagery  throughout 
the  tactical  battlefield  using  a  Ku-  and  Ka-satellite-based 
network.  The  Army  has  fielded  WIN-T  Increment  1  to  its 
operational  forces. 


Point  of  Presence 


Soldier  Network  Extension 


Tactical  Comms  Node 


1  -  Net-Centric  Waveform 

Antenna 

2  -  High-Band  Networking 

Waveform  Antenna 


-  Increment  2:  “Initial  Networking  On-the-Move”  provides 
command  and  control  on-the-move  down  to  the  company 
level  for  maneuver  brigades  and  implements  an  improved 
network  security  architecture.  WIN-T  Increment  2 
supports  on-the-move  communications  for  commanders 
with  the  addition  of  the  Point-of-Presence  (PoP)  and  the 
SNE  and  provides  a  mobile  network  infrastructure  with  the 
Tactical  Communications  Node  (TCN). 

-  Increment  3:  “Full  Networking  On-the-Move”  provides 
full  mobility  command  and  control  for  all  Army  field 
commanders,  from  theater  to  company  level.  Network 
reliability  and  robustness  are  enhanced  with  the  addition 
of  the  air  tier  transport  layer,  which  consists  of  networked 
airborne  communications  relays. 

-  Increment  4:  “Protected  Satellite  Communications 
On-the-Move”  includes  access  to  the  next  generation  of 
protected  communications  satellites  while  retaining  all 
previous  on-the-move  capabilities. 

Mission 

Commanders  at  theater  level  and  below  will  use  WIN-T  to: 

•  Integrate  satellite-based  communications  capabilities  into 
an  everything-over-lnternet  Protocol  network  to  provide 
connectivity,  while  stationary,  across  an  extended,  non-linear 
battlefield  and  at  remote  locations  (Increment  1). 


WIN-T 
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•  Provide  division  and  below  maneuver  commanders  with  Major  Contractor 

mobile  communications  capabilities  to  support  initial  General  Dynamics,  C4  Systems  -  Taunton,  Massachusetts 

command  and  control  on-the-move  (Increment  2). 

•  Provide  all  maneuver  commanders  with  mobile 
communications  capabilities  to  support  full  command 
and  control  on-the-move,  including  the  airborne  relay  and 
protected  satellite  communications  (Increments  3  and  4). 


Activity 

•  In  20 1 1 ,  the  Army  conducted  a  developmental  test  that 
assessed  a  W1N-T  Increment  2  network  containing  more  than 
70  communications  nodes.  The  Army  conducted  the  test  at 
Aberdeen  Proving  Ground,  Maryland,  and  the  contractor’s 
facility  in  Taunton,  Massachusetts. 

•  In  January  2012,  the  Army  conducted  cold  weather 
developmental  testing  at  the  Cold  Weather  Natural 
Environmental  Testing,  Fort  Greeley,  Alaska. 

•  In  February  2012,  the  Army  approved  a  revised  requirement 
that  lowered  WIN-T  Increment  2’s  reliability  requirement  by 
30-60  percent  based  upon  an  updated  operational  mission 
summary/mission  profile. 

•  In  May  2012,  the  Army  Test  and  Evaluation  Command 
(ATEC)  conducted  the  WIN-T  Increment  2  IOT&E  using  the 
following  units  employed  under  operationally  realistic  mission 
scenarios. 

-  2nd  Brigade,  1st  Armored  Division,  Fort  Bliss,  Texas,  and 
White  Sands  Missile  Range,  New  Mexico 

-  101st  Airborne  Division,  Fort  Campbell,  Kentucky 

-  Sustainment  Brigade,  Fort  Riley,  Kansas 

-  Network  Service  Center  -  Training,  Fort  Gordon,  Georgia 

•  The  WIN-T  Increment  2  IOT&E  included  a  fully  equipped 
brigade  and  division  headquarters  dispersed  over  a  wide 
geographic  area  to  assess  WIN-T  Increment  2’s  capability 
to  support  the  unit’s  at-the-halt  and  on-the-move  mission  in 
desert,  forest,  and  urban  terrain.  ATEC  conducted  IOT&E 
in  accordance  with  a  DOT&E-approved  Test  and  Evaluation 
Master  Plan  and  test  plan. 

•  On  September  2012,  the  DAE  chaired  the  WIN-T  Increment  2 
Defense  Acquisition  Board  (DAB)  to  consider  whether  to 
approve  the  system  for  full-rate  production  and  published 

an  Acquisition  Decision  Memorandum  (ADM)  detailing  the 
results.  DOT&E  provided  a  BLR1P  report  to  support  the 
DAB. 

Assessment 

•  During  the  2011  developmental  test,  the  Army’s 
developmental  test  efforts  assessed  that  WIN-T  Increment  2 
met  its  performance  requirements  under  benign  conditions. 
WIN-T  Increment  2  did  not  meet  its  reliability  requirements 
and  did  not  demonstrate  planned  reliability  growth. 

•  During  the  Army’s  January  2012  cold  weather  testing,  WIN-T 
Increment  2  met  its  cold  weather  requirements.  During 
testing,  the  PoP  demonstrated  reduced  bandwidth  throughput 


(up  to  10  percent)  when  the  configuration  item  switched 
between  terrestrial  HNW  and  satellite  NCW. 

•  DOT&E  assessed  the  following  WIN-T  Increment  2  items 
as  effective  based  upon  performance  demonstrated  during 
IOT&E: 

-  TCN.  The  TCN  improved  mission  performance  by 
sustaining  a  mission  command  network,  and  providing 
voice  and  data  access  to  the  network  while  on-the-move. 

-  PoP.  The  PoP  provided  voice  and  data  support  for 
commanders  while  on-the-move  and  at-the-halt. 

-  Network  Operations  and  Security  Center  (NOSC).  The 
NOSC  supported  the  unit’s  network  management  mission 
at  division  and  brigade,  but  needed  additional  Soldiers  and 
tools  at  battalion  and  company. 

-  Colorless  Core.  The  Colorless  Core  supported  multiple 
security  levels  and  improved  bandwidth  allocation. 

-  NCW.  Given  sufficient  satellite  bandwidth,  the  NCW 
connected  TCNs  and  PoPs  to  the  network  and  provided 
sufficient  data  flow  while  at-the-halt  and  on-the-move. 

-  Satellite  Tactical  Terminal  +  (STT+).  The  STT+ 
demonstrated  simultaneous  connections  of  its  satellite 
waveforms  in  support  of  the  unit’s  mission. 

-  Vehicle  Wireless  Package  (VWP).  The  VWP  was  useful 
at  the  brigade  and  division  levels,  but  should  be  installed 
in  different  vehicles  at  battalion  level  to  better  support 
command  post  movements. 

-  Joint  Gateway  Node  (JGN).  The  JGN  allows  WIN-T  to 
connect  to  a  variety  of  external  networks. 

-  Modular  Communications  Node  -  Basic  (MCN-B).  The 
MCN-B  allowed  the  unit  to  extend  subscriber  services 
from  an  adjacent  TCN. 

•  DOT&E  assessed  the  following  WIN-T  Increment  2  items  as 
not  effective  based  upon  performance  demonstrated  during  the 
IOT&E: 

-  SNE.  The  SNE  did  not  support  commanders  while 
on-the-move  but  served  as  an  alternate  communications 
means  while  at-the-halt. 

-  HNW.  The  HNW  terrestrial  line-of-sight  waveform 
demonstrated  poor  transmission  range  in  vegetation  and 
routing  problems  that  decreased  the  WIN-T  Increment  2 
network’s  performance. 

-  TR-T.  The  single  TR-T  employed  at  brigade  was  not  able 
to  keep  the  HNW  network  from  fragmenting  when  the  unit 
dispersed. 
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•  DOT&E  assessed  the  WIN-T  Increment  2  as  not  suitable  due 
to  poor  reliability  and  poor  maintainability: 

-  The  VWP  and  MCN-B  met  their  reliability  requirements. 
The  TCN,  PoP,  SNE,  TR-T,  and  NOSC  did  not  meet 
their  reliability  requirements.  The  WIN-T  Increment  2 
on-the-move  platforms  (TCN,  PoP,  and  SNE)  reliability 
estimates  demonstrated  during  IOT&E  are  less  than  one 
third  of  the  Army  requirement. 

-  The  VWP  and  JGN  met  their  maintainability  requirements, 
while  the  remaining  configuration  items  did  not  meet  the 
Mean  Time  to  Repair  requirement.  The  IOT&E  hosted 
twice  the  number  of  Field  Service  Representatives  to 
perform  maintenance  (relative  to  the  Army’s  support  plan). 
Even  with  increased  Field  Service  Representatives  present 
at  IOT&E,  repair  times  for  half  of  the  configuration  items 
took  two  to  four  times  longer  than  the  Army’s  requirement. 

•  Asa  result  of  IOT&E,  DOT&E  assessed  WIN-T  Increment  2 
as  not  survivable  due  to  significant  Information  Assurance 
vulnerabilities  that  would  degrade  a  unit’s  ability  to  succeed 
in  combat.  These  vulnerabilities  are  discussed  in  a  classified 
annex  to  the  DOT&E  BLRIP  report. 

•  On  September  26,  2012,  the  DAE  signed  an  ADM  that: 

-  Authorized  the  Army  to  procure  an  additional  538  WIN-T 
Increment  2  communication  nodes  as  a  second  Low-Rate 
Initial  Production. 

-  Directed  the  Army  to  conduct  an  FOT&E  to  demonstrate 
operational  effectiveness  and  suitability  of  the  SNE  and 
HNW. 

-  Provide  evidence  that  each  configuration  item  is  on 
track  per  approved  growth  curves  to  meet  reliability  and 
maintainability  requirements. 


•  The  Army  plans  to  conduct  an  FOT&E  during  NIE  13.2 
in  May  2013  to  demonstrate  that  WIN-T  Increment  2  has 
addressed  the  operational  effectiveness  and  suitability 
deficiencies  noted  in  the  DOT&E  BLRIP  report  as  directed  by 
the  September  ADM. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program 
successfully  addressed  one  of  the  three  FY10 
recommendations.  The  program  still  needs  to  correct 
deficiencies  identified  during  the  WIN-T  Increment  2  LUT, 
complete  requirements  documents  for  Increment  3,  update  the 
Increment  2  TEMP,  and  develop  an  Increment  3  TEMP. 

•  FY12  Recommendations.  The  Army  should: 

1 .  Create  a  reliability  growth  plan  to  improve  the  poor 
WIN-T  Increment  2  reliability  highlighted  during  IOT&E. 
Reliability  improvements  should  be  demonstrated  during  a 
future  operational  test  event. 

2.  Identify  the  root  causes  of  the  SNE  deficiencies,  correct  the 
poor  performance,  and  demonstrate  its  effectiveness  in  a 
future  operational  test  event. 

3.  Improve  HNW  and  NCW  to  address  deficiencies  noted 
during  the  IOT&E.  Waveform  improvements  should  be 
demonstrated  during  a  future  operational  test  event. 

4.  Complete  a  post  full-rate  production  TEMP  and  ensure 
funding  is  available  to  conduct  a  future  operational  test  to 
demonstrate  improvements  in  WIN-T  Increment  2. 
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Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion 

(A-RCI) 


Executive  Summary 

•  The  Navy  completed  FOT&E  of  Acoustic  Rapid  Commercial 
Off-the-Shelf  Insertion  (A-RCI)  Advanced  Processor  Build  09 
(APB-09)  in  early  FY12. 

•  DOT&E  issued  a  classified  FOT&E  report  for  the  A-RCI 
APB-09  system  in  November  2012,  and  found  the  APB-09 
system  provides  performance  similar  to  previous  APBs  (not 
improved  or  degraded). 

•  The  processing  and  display  for  the  Wide  Aperture  Array 
(WAA),  a  primary  focus  for  APB-09  software  upgrades, 
suffered  from  significant  technical  problems  that  were 
discovered  during  initial  operational  testing.  The  Navy 
developed  new  WAA  software  intended  to  fix  the  problems, 
conducted  limited  additional  development  testing,  and  issued 
the  new  software  to  the  fleet  without  operational  testing. 

•  The  Navy  is  completing  development  of  the  APB- 1 1  version 
and  operational  testing  is  planned  to  begin  in  FY13. 

System 

•  A-RCI  is  an  open-architecture  sonar  system  intended 
to  maintain  an  advantage  in  acoustic  detection  of  threat 
submarines. 

•  A-RCI  uses  legacy  sensors  and  replaces  central  processors 
with  commercial  off-the-shelf  (COTS)  computer  technology 
and  software.  The  program  includes  the  following: 

-  A  sonar  system  for  the  Virginia  class  submarine 

-  A  replacement  sonar  system  retrofitted  into  Los  Angeles, 
Ohio,  and  Seawolf  class  submarines 

-  Biannual  software  upgrades  (called  Acoustic  Processor 
Builds  (APBs))  and  hardware  upgrades  (called  Technology 
Insertions  (TIs)).  While  using  the  same  process  and 
nomenclature,  these  APBs  and  TIs  are  distinct  from  those 
used  in  the  AN/BYG-1  Combat  Control  System  program. 

•  The  Navy  intends  for  the  A-RCI  upgrades  to  provide  expanded 
capabilities  for  Anti-Submarine  Warfare  (ASW),  high-density 
contact  management,  and  mine  warfare,  particularly  in  littoral 
waters  and  against  diesel  submarines. 

•  A-RCI  processes  data  from  the  submarine’s  acoustic  arrays 
(i.e.,  spherical  array,  hull  array,  WAA,  and  high-frequency 
arrays)  along  with  the  submarine’s  two  towed  arrays  (i.e.,  the 
fat  line  array  consisting  of  the  TB-16  or  TB-34  and  the  thin 
line  array  consisting  of  the  TB-23  or  TB-29). 


•  The  APB-09  introduced  upgraded  processing  on  the  high 
frequency  array  and  the  ability  to  process  new  pulse  types. 

It  also  introduced  advanced  signal  processing  to  improve 
both  detection  and  display  performance  for  the  WAA.  The 
Navy  intends  for  the  new  software  to  improve  ASW  search 
through  attack  performance  with  new  enhanced  narrowband 
processing  algorithms,  to  improve  performance  avoiding 
threat  mines,  and  to  improve  situational  awareness  and  contact 
managements  for  all  missions. 

Mission 

The  Navy’s  intent  for  submarine  crews  equipped  with  the  A-RCI 

sonar  is  to  complete  the  following  submarine  force  missions: 

•  Search,  detect,  and  track  submarine  and  surface  vessels  in 
open-ocean  and  littoral  sea  environments  without  being 
counter-detected 

•  Search,  detect,  and  avoid  mines  and  other  submerged  objects 

•  Covertly  conduct  intelligence,  surveillance,  and 
reconnaissance 

•  Covertly  execute  Naval  Special  Warfare  missions 

•  Perform  under-ice  operations 

Major  Contractor 

Lockheed  Martin  Maritime  Systems  and  Sensors  -  Washington, 

District  of  Columbia 


Activity 

•  The  Navy  completed  FOT&E  of  A-RCI  APB-09  in  early 
FY12  in  accordance  with  a  DOT&E-approved  test  plan.  The 
Navy  combined  testing  with  the  AN/BYG-1  Combat  Control 


System,  the  Virginia  class  submarine,  and  the  Mk  48  Mod  6 
Advanced  Common  Torpedo  (ACOT)  and  Mk  48  Advanced 
Capability  (ADCAP)  Mod  7  Common  Broadband  Advanced 
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Sonar  System  (CBASS)  programs.  Coordinating  these  tests 
provided  testing  efficiencies  while  enabling  an  end-to-end 
evaluation  of  mission  performance. 

•  DOT&E  issued  a  classified  combined  test  report  for 
the  Virginia  class  submarine,  the  A-RCI  sonar,  and  the 
AN/BYG-1  Combat  Control  system  in  November  2012 
that  evaluated  the  effectiveness  and  suitability  of  the  A-RCI 
APB-09  system. 

•  The  Navy  began  drafting  a  Test  and  Evaluation  Master  Plan 
(TEMP)  for  the  APB- 1 1  and  APB- 1 3  A-RCI  variant  APBs, 
and  expects  to  issue  it  by  early  FY13.  As  part  of  these 
efforts,  DOT&E  requested  the  Navy  investigate  new  methods 
of  land-based  testing  and  onboard  simulated  target  injection 
methods  to  augment  at-sea  operational  tests.  Operational 
testing  of  the  APB- 11  variant  of  A-RCI  is  expected  to  begin 
in  FY13. 

Assessment 

•  The  DOT&E  classified  FOT&E  report  for  the  A-RCI  APB-09 
system  concluded  the  following  regarding  test  adequacy  and 
system  performance: 

-  Given  the  data  available  and  the  limitations  of  the  test, 
DOT&E  concluded  that  no  evidence  existed  to  change 
the  conclusions  about  mission  performance  from 
previous  reports  on  A-RCI  (not  improved  or  degraded). 
Specifically, 

■  For  ASW,  A-RCI  passive  sonar  capability  is 
effective  against  older  classes  of  submarines  in  some 
environments,  but  is  not  effective  in  all  environments  or 
against  modem  threats. 

■  A-RCI  is  not  effective  in  supporting  operator  situational 
awareness  and  contact  management  in  areas  of 
high-contact  density. 

■  A-RCI  high-frequency  mine  performance  is  not 
effective  for  some  types  of  minefields,  but  meets 
threshold  requirements  against  some  mine  types  under 
certain  environmental  conditions. 

-  Testing  to  examine  AS  W- Attack  and  situational 
awareness  in  a  high-surface-ship  density  environment 
was  adequate  for  the  system  tested  but  not  adequate  for 
the  software  version  fielded.  The  A-RCI  processing  for 
the  WAA  suffered  from  system  development  and  software 
problems,  which  led  to  poor  performance  during  the 
operational  testing.  The  Navy  investigated  this  issue 
after  operational  testing  was  completed  and  subsequently 
issued  software  intended  to  fix  the  problems.  The  Navy 
conducted  some  limited  developmental  testing  to  confirm 
functionality;  however,  the  Navy  has  not  conducted 
operational  testing  to  evaluate  the  updated  software  or 
potential  changes  to  mission  performance. 

-  Test  execution  to  examine  the  Precision  Underwater 
Mapping  capability  was  not  adequate. 

-  Several  mine  shapes  in  the  Navy’s  training  minefield 
used  for  the  operational  testing  were  severely  corroded, 


damaged,  or  buried.  DOT&E  considered  the  condition 
of  the  mine  shapes  in  evaluating  high-frequency  sonar 
detection  performance  and  assessed  the  testing  was 
adequate  for  some  types  of  threat  minefields. 

-  A-RCI  continues  to  be  not  suitable  due  to  problems  with 
reliability  and  operator  training.  For  APB-09,  the  Navy 
lowered  the  reliability  requirements  below  the  previously 
measured  APB  reliability.  The  Navy  also  refocused  the 
system’s  new  reliability  requirements  on  the  minimum 
system  functions  necessary  for  at-sea  operations  vice 
what  system  functions  are  required  for  the  mission.  While 
APB-09  met  the  revised  lower  reliability  requirements, 
the  APB-09  system’s  reliability  was  significantly  below 
the  sonar  reliability  demonstrated  on  Virginia  class 
submarines  during  the  IOT&E.  For  other  submarine 
classes,  measured  sonar  APB  reliability  is  statistically 
unchanged  from  previous  APBs.  After  operational 
testing,  the  Navy  reported  that  software  changes  were 
made  to  the  APB-09  software  intended  to  fix  the 
reliability  problems  identified  in  testing;  these  changes 
have  not  been  evaluated. 

-  The  Navy  revised  the  Capability  Development 
Document/Capability  Production  Document  performance 
requirements  for  A-RCI  APB-09.  The  revised 
requirements  metric  focuses  development  and  testing  on 
the  time  between  the  A-RCI  system  displaying  acoustic 
energy  and  the  operator  identifying  the  target.  This 
new  metric  favors  shorter  range  detections;  therefore,  a 
poorly-performing  sonar  system  (shorter  range  detections) 
has  the  potential  of  scoring  better  than  a  long-range 
detecting  system. 

•  Due  to  the  A-RCI  biannual  software  and  hardware 
development  cycle,  the  Navy  generates  and  approves  the 
A-RCI  requirements  documents  and  TEMPs  in  parallel  with 
APB  development  and  installation.  As  a  result,  the  fleet 
assumes  additional  risk,  since  most  operational  testing  is  not 
completed  before  the  system  is  initially  deployed. 

•  The  Navy’s  schedule-driven  process  prevents  operational  test 
results  from  directly  supporting  development  of  the  follow-on 
APBs.  For  example,  the  Navy  completed  operational  testing 
of  the  A-RCI  APB-09  system  in  early  FY12.  Due  to  the 
combination  of  the  late  completion  of  testing  and  the  Navy’s 
practice  of  issuing  an  updated  version  every  two  years,  data 
from  APB-09  operational  testing  could  not  be  included  in  the 
development  of  APB-11. 

•  The  development  and  testing  of  APB-09  is  an  example  of 
the  high-risk  of  schedule-driven  development  and  fielding; 
operational  testing  revealed  significant  performance  failures 
with  the  WAA  that  were  not  discovered  in  developmental 
testing.  Although  the  Navy  issued  new  software  intended 
to  fix  the  identified  problems  after  the  operational  test, 
submarines  deployed  with  a  version  of  A-RCI  that  the  Navy 
has  not  operationally  tested. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  making 

progress  in  addressing  23  of  the  30  recommendations 

contained  in  the  APB-00  to  APB-07  OT&E  reports.  The 

significant  remaining  unclassified  recommendations  are: 

1 .  Improve  the  detection  and  localization  performance 
for  submarines  operating  in  high-density  surface  ship 
environments.  Consider  investing  in  automation  that 
will  assist  the  operator  in  processing  the  large  amount  of 
constantly  changing  contact  data  and  determining  which 
contacts  pose  an  immediate  collision  or  counter-detection 
threat. 

2.  Improve  operator  training  such  that  operators  understand 
and  effectively  employ  new  APB  functionality  when 
fielded.  Many  of  the  newly  introduced  features  in  APBs 
that  were  designed  to  improve  mission  performance  were 
not  used  consistently  during  the  test. 

3.  Evaluate  the  covertness  of  the  high-frequency  sonar  during 
a  future  submarine-on-submarine  test. 

4.  Investigate  the  software  reliability  problems  and  institute 
measures  to  improve  system  reliability. 

5.  Conduct  future  ASW  testing  against  a  high-end 
diesel-electric  submarine  (SSK)  target  to  enable  a  more 
complete  assessment  of  A-RCI  and  BYG-1  performance. 

•  FY12  Recommendations.  The  Navy  should: 

1.  Consolidate  the  A-RCI  and  AN/BYG-1  TEMPs  into  an 
Undersea  Enterprise  Capstone  document. 

2.  Re-evaluate  operational  effectiveness  and  suitability  of 
A-RCI  on  a  submarine  with  the  new  WAA  software. 

3.  Address  the  21  recommendations  for  the  A-RCI  and  the 
AN/BYG-1  systems  contained  in  DOT&E’s  FOT&E  report 


on  A-RCI  APB-09.  Some  of  the  significant  unclassified 
recommendations  are: 

-  The  Navy  should  improve  its  developmental  testing 
processes  and  metrics  used  to  determine  if  a  system 
potentially  improves  effectiveness  and  suitability  and  to 
ensure  that  the  system  is  ready  for  operational  testing  and 
subsequent  fielding. 

-  Search  rate  is  an  important  metric  for  evaluating  the 
ASW  search  performance.  Recently,  the  Navy  issued 
revised  requirements  for  A-RCI  and  did  not  update  the 
platform  search  rate  metric.  The  new  metrics,  which 
assess  the  difference  between  the  initial  operator  and 
post-test  laboratory  detection  times,  could  improperly 
result  in  a  poorly  performing  system  (short-range 
detections)  scoring  better  than  a  long-range  detecting 
sonar  system.  DOT&E  recommends  the  Navy 
re-evaluate  the  new  metrics  to  improve  their  robustness 
under  varying  environmental  conditions  and  to  focus  on 
earlier  and  longer  range  operator  detections.  Also,  the 
Navy  should  update  the  operational  and  environmental 
conditions  for  the  search  rate  metric  to  reflect  current 
threats  and  threat  areas. 

-  If  future  minefield  testing  requires  the  use  of  existing 
fleet  training  minefields,  a  minefield  video  survey  to 
evaluate  the  condition  and  location  of  the  mines  should 
be  conducted  prior  the  decision  to  use  the  minefield  for 
testing. 
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Aegis  Modernization  Program 


Executive  Summary 

•  The  Navy  completed  operational  testing  of  Aegis  guided 
missile  cruisers  (CG-52  through  CG-58)  upgraded  with 
Aegis  Warfare  System  (AWS)  Advanced  Capability 
Build  2008  (ACB08)  and  Aegis  guided  missile  destroyers 
(DDG-103  through  DDG-112)  upgraded  with  AWS 
Baseline  7. 1R  in  1QFY 12.  The  Navy  did  not  scope  the 
ACB08/Baseline  7. 1R  testing  to  provide  a  comprehensive 
evaluation  of  effectiveness  and  suitability.  The  purpose  of 
these  tests  was  to  verify  the  recent  updates  did  not  degrade 
AWS  performance.  Based  upon  operational  testing  in  FY12, 
the  ACB08/Baseline  7.1R  AWS  performance  is  consistent 
with  the  performance  of  previous  AWS  versions,  which 
DOT&E  assessed  as  effective  and  suitable. 

•  Operational  testing  of  the  ACB08/Baseline  7. 1R  did  not 
support  a  full  assessment  of  Aegis  effectiveness  in  the  area 
defense  mission.  The  introduction  of  AWS  Baseline  9 

in  FY14  introduces  new  requirements  and  fields  new 
performance  capabilities  that  are  intended  to  support  such  an 
assessment. 

•  The  analysis  of  test  data  collected  during  AWS 
Baseline  7. 1R  operational  testing  and  the  remaining  air 
defense  and  suitability  portions  of  AWS  ACB08  operational 
testing  is  ongoing.  DOT&E  will  issue  a  formal  test  report  in 
2QFY13. 

System 

•  The  Navy’s  Aegis  Modernization  program  provides  updated 
technology  and  systems  for  existing  Aegis  guided  missile 
cruisers  (CG-47)  and  destroyers  (DDG-51).  This  planned, 
phased  program  provides  similar  technology  and  systems  for 
new  destroyers. 

•  The  AWS,  carried  on  DDG-51  guided  missile  destroyers 
and  CG-47  guided  missile  cruisers,  integrates  the  following 
components: 

-  AWS  AN/SPY-1  three-dimensional  (range,  altitude,  and 
azimuth)  multi-function  radar 

-  SQQ-89  Undersea  Warfare  suite  that  includes  the 
AN/SQS-53  sonar,  SQR-19  passive  towed  sonar  array 
(DDG-51  through  DDG-78,  CG-52  through  CG-73),  and 
the  SH-60B  or  MFI-60R  Helicopter  (DDG-79  and  newer 
have  a  hangar  to  allow  the  ship  to  carry  and  maintain  its 
own  helicopter) 

-  Close-In  Weapon  System 

-  Five-inch  diameter  gun 

-  Harpoon  anti-ship  cruise  missiles  (DDG-51  through 
DDG-78,  CG-52  through  CG-73) 

-  Vertical  Launch  System  that  can  launch  Tomahawk 
land-attack  missiles,  Standard  surface-to-air  missiles, 


Evolved  SeaSparrow  Missiles,  and  Vertical  Launch 
Anti-Submarine  Rocket  missiles 

•  The  AWS  on  Baseline  2  Aegis  guided  missile  cruisers 
(CG-52  through  CG-58)  was  upgraded  with  commercial 
off-the-shelf  hardware  running  the  AWS  software  ACB08. 

•  The  AWS  on  new  construction  Aegis  guided  missile 
destroyers  (DDG-103  through  DDG-112)  is  Baseline  7. 1R. 

Mission 

The  Joint  Force  Commander/Strike  Group  Commander 

employs  AWS-equipped  DDG-51  guided  missile  destroyers  and 

CG-47  guided  missile  cruisers  to: 

•  Conduct  area  and  self-defense  Anti-Air  Warfare  in  defense 
of  the  Strike  Group 

•  Conduct  Anti-Surface  Warfare  and  Anti-Submarine  Warfare 

•  Conduct  Strike  Warfare  when  armed  with  Tomahawk 
missiles 

•  Conduct  offensive  and  defensive  warfare  operations 
simultaneously 

•  Operate  independently  or  with  Carrier  or  Expeditionary 
Strike  Groups,  as  well  as  with  other  joint  or  coalition 
partners 

Major  Contractors 

•  General  Dynamics  Marine  Systems  Bath  Iron  Works  -  Bath, 
Maine 

•  Northrop  Grumman  Shipbuilding  -  Pascagoula,  Mississippi 

•  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Moorestown,  New  Jersey 
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Activity 

•  The  Navy  conducted  all  portions  of  the  planned  operational 
test  of  AWS  ACB08  in  FY10,  with  the  exception  of  air  defense 
and  suitability  testing,  which  it  completed  in  1QFY12.  All 
testing  was  conducted  in  accordance  with  a  DOT&E-approved 
Test  and  Evaluation  Master  Plan  and  test  plan. 

•  The  Navy  conducted  all  portions  of  the  planned  operational 
test  of  AWS  Baseline  7.1R  concurrently  with  the  last  phase  of 
ACB08  testing  in  1QFY12. 

•  The  Navy  repaired  and  examined  critical  software  faults 
discovered  during  operational  testing  of  AWS  ACB08  in 
4QFY12.  This  testing  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan. 

•  The  Navy  continued  to  deploy  AWS  ACB08-equipped  cruisers 
and  AWS  Baseline  7.1R-equipped  destroyers  in  FY12  in 
advance  of  operational  testing. 

•  The  Navy  is  preparing  an  update  to  the  Test  and  Evaluation 
Master  Plan  to  incorporate  AWS  baseline  ACB  2012  (ACB12). 
ACB 12  will  provide  existing  and  new  construction  Aegis 
destroyers  an  Integrated  Air  and  Missile  Defense  (IAMD) 
capability  that  includes  Ballistic  Missile  Defense  (BMD). 
Current  plans  will  also  provide  an  enhanced  Air  Defense 
capability  to  selected  Aegis  cruisers  without  BMD  (CGs-59, 
-60,  -62,  and  -71). 

Assessment 

•  The  Navy  did  not  scope  the  ACB08/Baseline  7. 1R  testing 
to  provide  a  comprehensive  evaluation  of  effectiveness  and 
suitability.  The  purpose  of  these  tests  was  to  verify  that  the 
recent  updates  did  not  degrade  AWS  performance.  Based 
upon  operational  testing  in  FY12,  the  ACB08/Baseline  7.1R 
AWS  performance  is  consistent  with  the  performance  of 
previous  AWS  versions,  which  DOT&E  assessed  as  effective 
and  suitable.  Analysis  of  the  test  data  indicates  the  following: 

-  The  Air  Warfare  performance  of  Aegis  cruisers  and 

destroyers,  in  self-defense  posture,  is  consistent  with  that 


of  ships  with  earlier  Aegis  baselines.  Testing  did  not  focus 
on  the  area  defense  capability  of  the  ACB08/7.1  RAWS. 
The  Navy  intends  to  conduct  an  assessment  of  area  defense 
effectiveness  with  the  introduction  of  ACB12/AWS 
Baseline  9  in  FY14. 

-  The  Undersea  Warfare  performance  of  Aegis  cruisers  with 
AWS  ACB08  is  consistent  with  that  of  ships  with  earlier 
Aegis  baselines. 

-  The  Surface  Warfare  performance  of  Aegis  cruisers  and 
destroyers  is  consistent  with  that  of  ships  with  earlier 
Aegis  baselines.  As  previously  assessed,  Aegis  ships  have 
limited  ability  to  counter  high-speed  surface  threats  in 
littoral  waters. 

•  Operational  testing  of  ACB08/Baseline  7.1R  in  a  multi-ship, 
Cooperative  Engagement  Capability  environment  revealed 
shortcomings  in  crew  proficiency  and  training  not  observed  in 
earlier  testing  that  adversely  affected  Air  Warfare  performance. 

•  The  analysis  of  data  collected  during  follow-on  testing  of 
AWS  ACB08  is  still  in  progress.  DOT&E  will  report  on  the 
corrective  action  in  the  ACB08  test  report  in  2QFY13. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
completed  all  FY11  recommendations. 

•  FY 12  Recommendations.  The  Navy  should: 

1.  Continue  to  improve  Aegis  ships’  capability  to  counter 
high-speed  surface  threats  in  littoral  waters. 

2.  Synchronize  future  baseline  operational  testing  and 
reporting  with  intended  ship-deployment  schedules  to 
ensure  that  testing  and  reporting  is  completed  prior  to 
deployment. 

3.  Ensure  Aegis  crews  are  proficient  in  operation  of  the  AWS 
when  in  a  multi-ship,  network  environment. 
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AGM-88E  Advanced  Anti-Radiation  Guided  Missile 

(AARGM)  Program 


Executive  Summary 

•  The  AGM-88E  Advanced  Anti-Radiation  Guided  Missile 
(AARGM)  program  completed  IOT&E  and  conducted  an 
additional  Verification  of  Correction  of  Deficiencies  (VCD) 
test  period  during  FY12. 

•  During  FY11/12  operational  testing,  the  Navy  completed 

185  sorties,  accumulating  558  hours  of  missile  operating  time. 
The  Navy  executed  396  captive-carry  runs  and  fired  a  total  of 
12  missiles  at  actual  and  simulated  threat  targets  and  emitters. 

•  DOT&E  published  a  classified  IOT&E  report  in  4QFY12. 

•  AARGM  is  operationally  suitable. 

•  AARGM  is  not  operationally  effective.  Although  AARGM 
has  the  potential  to  eventually  provide  some  improved  combat 
capability  against  enemy  air  defenses,  the  current  weapon 
configuration  has  multiple  performance  shortfalls  that  largely 
negate  its  ability  to  accomplish  its  mission. 

•  The  Navy  expects  the  AARGM  Block  1  Upgrade  to 
address  significant  performance  shortfalls  and  provide  Full 
Operational  Capability,  with  the  associated  FOT&E  scheduled 
to  commence  in  FY14. 

•  Due  to  deferred  capabilities  and  IOT&E  deficiencies,  DOT&E 
anticipates  that  AARGM  FOT&E  requirements  exceed  the 
program’s  currently  allocated  resources  for  operational  testing. 
A  shortage  of  AARGM  telemetry  kits  is  already  identified  as  a 
potential  hindrance  to  adequate  Block  1  testing. 

System 

•  The  AGM-88E  AARGM  is  the  follow-on  to  the  AGM-88B/C 
High-Speed  Anti-Radiation  Missile  (HARM)  using  a  modified 
HARM  body  and  fins.  The  Navy  intends  to  employ  AARGM 
on  F/A-18C/D/E/F  and  EA-18G  platforms. 

•  AARGM  Block  0,  intended  for  Initial  Operational  Capability, 
incorporates  digital  Anti-Radiation  Homing  (ARH),  GPS,  and 
Millimeter  Wave  (MMW)  guidance,  and  a  Weapon  Impact 
Assessment  transmitter. 

-  ARH  improvements  over  HARM  include  an  increased 
field  of  view  and  larger  frequency  range. 

-  The  GPS  allows  position  accuracy  in  location,  time,  and 
weapon  impact  assessment  transmissions. 


-  MMW  radar  technology  allows  target  discrimination  and 
guidance  during  the  terminal  flight  phase. 

•  The  Navy  expects  the  AARGM  Block  1  Upgrade  to  deliver 
Full  Operational  Capability,  including  Block  0  capability 
improvements,  as  well  as  an  Integrated  Broadcast  Service 
Receiver  (enables  reception  of  national  broadcast  data),  and 
software  changes  to  provide  deferred  capability  requirements 
and  address  deficiencies  identified  during  IOT&E. 

Mission 

Commanders  employ  aircraft  equipped  with  AARGM  to  conduct 
pre-planned,  on-call,  and  time-sensitive  reactive  anti-radiation 
targeting  for  the  suppression,  degradation,  and  destruction  of 
radio  frequency  enabled  surface-to-air  missile  defense  systems. 
Commanders  receive  real-time  Weapons  Impact  Assessments 
from  AARGM  via  a  national  broadcast  data  system. 

Major  Contractor 

Alliant  Techsystems,  Defense  Electronics  Systems 
Division  -  Woodland  Hills,  California 


Activity 

•  In  2QF  Y 10,  the  Navy  issued  a  change  to  the  AARGM 
Capability  Production  Document  (CPD)  due  to  limitations 
discovered  during  developmental  testing.  This  CPD  change 
delayed  the  start  of  IOT&E  until  3QFY10  to  allow  correction 
of  system  deficiencies;  deferred  a  Key  Performance  Parameter 


target  requirement  to  FOT&E;  and  clarified  the  acceptable 
target  environment  and  reactive  targeting  constraints  for 
IOT&E. 

•  In  accordance  with  the  DOT&E -approved  Operational  Test 
Plan  (OTP),  the  Navy’s  Commander,  Operational  Test  and 
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Evaluation  Force  (COTF)  commenced  AARGM  IOT&E 
in  June  2010,  but  in  September  2010,  the  Navy  de-certified 
AARGM  from  IOT &E  after  the  program  suffered  six 
operational  mission  failures  during  initial  captive-carry  flight 
tests. 

•  The  AARGM  program  spent  most  of  FY 1 1  correcting 
hardware  and  software  deficiencies  discovered  in 
developmental  testing  and  during  its  first  IOT&E  attempt.  In 
July  2011,  DOT&E  approved  an  updated  AARGM  OTP  and 
the  Navy  re-initiated  dedicated  IOT&E  in  August  2011. 

•  Immediately  following  the  restart  of  IOT&E,  COTF  initiated 
changes  to  the  OTP  that  were  driven  by  the  2010  CPD  change 
and  deficiencies  identified  since  the  first  IOT&E  attempt 
was  terminated.  DOT&E  approved  the  requested  changes  in 
October  2011. 

•  The  Navy  postponed  two  live-missile  test  events  scheduled 
for  October  2011  due  to  an  emergent  anomaly  that  caused  a 
communication  failure  between  the  AARGM  Guidance  and 
Control  Sections  (GS/CS).  The  Navy  identified  a  short-term 
solution  for  this  problem  and  conducted  the  live-missile  events 
during  2QFY12. 

•  Immediately  preceding  a  February  2012  test  event  involving 
two  other  live-missile  shots,  the  Navy  notified  DOT&E  that 
the  planned  threat  scenario  would  likely  result  in  mission 
failure  due  to  a  classified  AARGM  deficiency  (details 
available  in  the  classified  DOT&E  IOT&E  report).  Without 
DOT&E  consent,  the  Navy  modified  the  approved  test 
scenario  to  alleviate  the  classified  deficiency  and  proceeded 
with  live-missile  testing.  DOT&E  disagreed  with  the  adjusted 
threat  representation  and  subsequently  assessed  these  events  as 
operational  failures. 

•  COTF  completed  IOT&E  events  in  2QFY12,  with  initial 
results  indicating  AARGM  was  neither  operationally  effective 
nor  operationally  suitable. 

•  The  Navy  subsequently  completed  software  changes  to  address 
the  two  most  significant  deficiencies  identified  during  IOT&E 
(GS/CS  communication  failure  and  a  classified  performance 
shortfall)  and  subsequently  conducted  a  VCD  test  period 
during  3QFY12.  DOT&E  approved  the  VCD  test  plan  and 
included  the  associated  data  in  its  IOT&E  assessment. 

•  During  operational  testing,  the  Navy  fired  a  total  of  12  missiles 
at  actual  and  simulated  threat  targets  and  emitters.  As  required 
by  DOT&E,  low-rate  initial  production  (LR1P)  missiles  were 
used  for  all  events. 

•  In  accordance  with  DOT&E-approved  test  plans,  the  Navy 
completed  185  operational  test  sorties,  accumulating  558  hours 
of  missile  operating  time.  These  totals  include  sorties 

and  hours  accrued  during  integrated  testing  that  DOT&E 
considered  operationally  representative,  with  the  exception  of 
the  modified  threat  scenario  mentioned  above. 

•  DOT&E  published  a  classified  IOT&E  report  in  4QFY12. 

•  The  Navy’s  Milestone  Decision  Authority  (MDA)  conducted  a 
Full-Rate  Production  (FRP)  Decision  Review  during  4QFY12. 
At  that  review,  the  Navy  authorized  AARGM  Block  0  for 
FRP;  however,  only  the  first  lot  of  FRP  missiles  was  approved. 


Due  to  AARGM  operational  effectiveness  concerns,  the  MDA 
stipulated  that  a  review  of  FRP  and  Block  1  progress  shall 
occur  before  the  end  of  FY13.  The  MDA  intends  to  base  its 
authorization  for  another  FRP  lot  on  this  follow-on  review, 
even  though  no  additional  operational  testing  is  planned  during 
FY13. 

•  The  AARGM  program  hosted  an  initial  T&E  Working-level 
Integrated  Product  Team  meeting  in  late  4QFY12  to  start 
identifying  and  coordinating  AARGM  Block  1  FOT&E 
requirements.  Additional  coordination  meetings  are  planned 
for  1QFY13. 

Assessment 

•  AARGM  Block  0  testing  was  adequate  to  support  an 
evaluation  of  the  weapon  system’s  operational  effectiveness 
and  operational  suitability.  With  the  exception  of  the  modified 
threat  scenario  mentioned  above,  the  Navy  completed  testing 
in  accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan,  OTP,  and  VCD  test  plan. 

•  AARGM  Block  0  is  operationally  suitable.  Although  the 
weapon  demonstrated  poor  reliability  during  IOT&E,  the 
program  addressed  the  primary  deficiency  affecting  reliability 
and  satisfactorily  demonstrated  this  during  the  VCD  test 
period. 

•  AARGM  Block  0  is  not  operationally  effective.  Although 
AARGM  has  the  potential  to  eventually  provide  some 
improved  combat  capability  against  enemy  air  defenses, 
the  weapon  as  tested  has  multiple  deficiencies.  The  details 
of  these  deficiencies  are  detailed  in  the  classified  DOT&E 
IOT&E  report.  The  numerous  performance  shortfalls 
identified  largely  negate  AARGM’s  ability  to  accomplish  its 
primary  mission. 

•  The  Navy  expects  AARGM  Block  1  Upgrade  to  address 
significant  performance  shortfalls  and  provide  Full  Operational 
Capability,  with  the  associated  FOT&E  scheduled  to 
commence  in  FY14.  DOT&E  anticipates  that  Block  1  FOT&E 
captive-carry  and  live-missile  firing  requirements  will  exceed 
the  program’s  currently  allocated  resources  for  operational 
testing.  Operational  testers  have  already  identified  a  shortage 
of  AARGM  telemetry  kits  as  a  potential  hindrance  to  adequate 
Block  1  testing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  two  FY  1 1 
recommendations  remain  valid  and  open.  DOT&E  requires 
FOT&E  to  verify  the  correction  of  integrated  testing  and 
IOT&E  deficiencies  and  to  adequately  assess  previously 
deferred  capabilities. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Limit  FRP  quantities  until  operational  effectiveness  is 
properly  demonstrated  during  AARGM  Block  1  FOT&E 
and  documented  in  a  DOT&E  report. 

2.  Ensure  adequate  inventory  of  AARGM  telemetry  kits  are 
available  for  AARGM  FOT&E. 
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AIM-9X  Air-to-Air  Missile  Upgrade 


Executive  Summary 

•  The  Navy  completed  AIM-9X  Block  II  Integrated  Testing 
in  March  2012.  The  Navy  assessed  8  of  12  missile  shots 
conducted  before  the  Operational  Test  Readiness  Review 
(OTRR)  as  “hits.”  Developmental  testing  also  included  83 
captive-carry  missions  with  telemetry  analyzed  in  detail,  and 
more  than  3,700  captive-carry  hours  for  quantifying  reliability. 

•  The  Navy  and  Air  Force  began  IOT&E  on  April  27,  2012. 
During  operational  testing  to  date,  the  Navy  has  completed 
10  of  22  planned  captive-carry  events  and  2  of  9  live  missile 
shots.  The  Air  Force  has  completed  10  of  22  captive-carry 
flights  and  4  of  8  live  missile  shots.  Four  of  the  six  live 
missile  shots  have  passed  within  lethal  radius  of  the  target. 

The  Services  plan  to  complete  IOT&E  in  July  2013. 

•  As  of  mid-November  2012,  the  Services  had  accomplished 
5,460  total  captive-carry  hours  and  had  23  failures  resulting  in 
a  Mean  Time  Between  Captive-Carry  Failure  (MTBCCF)  of 
237  hours.  The  current  system  reliability  is  slightly  below  the 
value  on  the  reliability  growth  curve  consistent  with  reaching 
the  requirement  of  500  hours  MTBCCF  at  80,000  hours. 

System 

•  AIM-9X  is  the  latest  generation  short-range,  heat-seeking, 
air-to-air  missile.  The  currently  fielded  version  of  the  missile 
is  AIM-9X  Block  I,  Operational  Flight  Software  (OFS)  8.220, 
which  includes  limited  lock-on-after-launch,  full  envelope 
off-boresight  capability  without  a  helmet-mounted  cueing 
system,  and  improved  flare  rejection  performance. 

•  AIM-9X  is  highly  maneuverable,  day/night  capable,  and 
includes  the  warhead,  fuze,  and  rocket  motor  from  the 
previous  AIM-9M  missile. 

•  AIM-9X  added  a  new  imaging  infrared  seeker,  vector 
controlled  thrust,  digital  processor,  and  autopilot. 

•  F-15C/D,  F-16C/D,  and  F/A-18C/F  aircraft  can  carry  the 
AIM-9X. 

•  The  AIM-9X  Block  II  is  the  combination  of  AIM-9X-2 
hardware  and  OFS  9.3. 

-  AIM-9X-2  is  the  latest  hardware  version  and  is  designed 
to  prevent  parts  obsolescence  and  provide  processing 
capability  for  the  OFS  9.3  upgrade.  The  AIM-9X-2  missile 


includes  a  new  processor,  a  new  ignition  battery  for  the 
rocket  motor,  an  electronic  ignition  safety/arm  device,  and 
the  DSU-41/B  Active  Optical  Target  Detector  (AOTD) 
fuze/datalink  assembly. 

-  OFS  9.3  is  a  software  upgrade  that  is  intended  to  add 
trajectory  management  to  improve  range,  datalink  with  the 
launching  aircraft,  improved  lock-on-after-launch,  target 
re-acquisition,  and  improved  fuzing. 

Mission 

Air  combat  units  use  the  AIM-9X  to: 

•  Conduct  short-range  offensive  and  defensive  air-to-air  combat 

•  Engage  multiple  enemy  aircraft  types  with  passive  infrared 
guidance  in  the  missile  seeker 

•  Seek  and  attack  enemy  aircraft  at  large  angles  away  from  the 
heading  of  the  launch  aircraft 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  The  Navy  completed  AIM-9X  Block  II  (AIM-9X-2 
with  OFS  9.3)  developmental  testing  in  March  2012. 
Developmental  testing  consisted  of  12  free  flight  shots,  4 
of  which  were  conducted  with  the  final  software  version 
(OFS  9.311).  It  also  included  83  captive-carry  missions  with 


telemetry  analyzed  in  detail,  and  more  than  3,700  captive-carry 
hours  for  quantifying  reliability. 

•  The  Navy  conducted  an  OTRR  in  April  2012,  and 

certified  readiness  for  AIM-9X  Block  II  IOT&E  under  the 
DOT&E-approved  June  2011  TEMP  and  March  2012  test  plan. 
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•  The  Navy  and  Air  Force  began  IOT&E  in  April  2012.  During 
operational  testing  to  date,  the  Navy  has  completed  10  of  22 
planned  captive-carry  events  and  2  of  9  live  missile  shots.  The 
Air  Force  has  completed  10  of  22  captive-carry  flights  and  4  of 
8  live  missile  shots. 

•  The  Services  plan  to  complete  IOT&E  in  July  2013,  with 
a  Full-Rate  Production  decision  in  April  2014  and  Initial 
Operational  Capability  planned  for  September  2014. 

Assessment 

•  During  developmental  testing,  9  of  12  total  missile  shots 
guided  to  within  lethal  radius  of  the  drone.  One  of  those  nine 
missiles  did  not  receive  a  fuze  pulse,  resulting  in  no  detonation 
within  proximity  of  the  drone.  Therefore,  the  Navy  assessed  8 
of  12  shots  conducted  before  the  OTRR  as  “hits.” 

•  All  captive-carry  missions  were  nominal,  but  the  Air  Force 
highlighted  one  performance  discrepancy  with  AIM-9X 
Block  II  Helmet-less  High  Off-Boresight  (HHOBS) 
performance.  It  is  possible  that  Block  II  is  slower  to  acquire 
targets  in  HHOBS  than  Block  I.  The  Capability  Production 
Document  (CPD)  requires  the  AIM-9X  Block  II  performance 
be  equal  to  or  better  than  baseline  AIM-9X  performance. 

•  At  the  OTRR,  reliability  was  232  hours  MTBCCF  and  is 
projected  to  reach  316  hours  at  the  end  of  IOT&E.  As  of 
mid-November  2012,  the  Services  had  accomplished  5,460 
total  captive-carry  hours  and  had  23  failures  resulting  in 
an  MTBCCF  of  237  hours.  The  current  system  reliability 
is  slightly  below  the  value  on  the  reliability  growth  curve 
consistent  with  reaching  the  requirement  of  500  hours 


MTBCCF  at  80,000  hours.  DOT&E  will  track  reliability 
through  IOT&E. 

•  All  Navy  and  Air  Force  IOT&E  captive-carry  sorties 
to  this  point  have  achieved  mission  objectives.  As  of 
November  15,  2012,  4  of  6  live  missile  shots  have  been 
successful.  The  AIM-9X  Block  II  did  not  achieve  a  hit  on  the 
fourth  shot  by  the  Air  Force,  attempted  in  September  2012.  It 
was  a  lock-on-after-launch  shot,  which  initially  locked  on  the 
target  but  then  lost  track  and  did  not  re-acquire.  The  Navy’s 
first  shot,  attempted  in  October  2012,  was  also  unsuccessful. 

It  was  another  lock-on-after-launch  shot  that  did  not  acquire 
the  target.  Data  results  are  still  pending  for  both  unsuccessful 
attempts.  To  date,  the  Air  Force  and  Navy  have  not  reported 
any  weapon  system  deficiencies  during  IOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
addressed  previous  recommendations. 

•  FY12  Recommendation. 

1.  The  Navy  should  address  the  Air  Force’s  concern  of 
HHOBS  performance.  It  shoidd  gather  data  to  verify  or 
disprove  AIM-9X  Block  II  performance  slip  during  HHOBS 
performance.  If  Block  II  performance  is  worse  than  that 
of  Block  1  in  HHOBS,  consider  a  software  modification  to 
improve  HHOBS  performance  and  comply  with  the  CPD 
requirement  of  achieving  performance  equal  to  or  greater 
than  Block  I. 
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AN/BYG-1  Combat  Control  System 


Executive  Summary 

•  The  Navy  completed  FOT&E  of  AN/BYG-1  Advanced 
Processor  Build  09  (APB-09)  in  early  FY12. 

•  DOT&E  issued  a  classified  FOT&E  report  for  the  AN/BYG-1 
APB-09  system  in  November  2012  and  found  the  APB-09 
system  provides  performance  similar  to  previous  APBs  (not 
improved  or  degraded). 

•  The  processing  and  display  for  the  Wide  Aperture  Array 
(WAA)  suffered  from  significant  technical  problems  that 
were  discovered  during  operational  testing.  The  WAA  is 
a  sonar  input  to  the  BYG-1  intended  to  provide  bearing 
and  ranges  data  to  the  system.  The  Navy  developed  new 
software  intended  to  fix  the  WAA  problems,  conducted  limited 
development  testing,  and  issued  the  new  software  to  the  fleet 
without  further  operational  testing. 

•  The  Navy  is  completing  development  of  the  APB- 1 1  version 
and  operational  testing  is  planned  to  begin  in  FY13. 

System 

•  AN/BYG-1  is  an  open-architecture  submarine  combat  control 
system  for  analyzing  and  tracking  submarine  and  surface 
ship  contacts,  providing  situational  awareness,  as  well  as  the 
capability  to  target  and  employ  torpedoes  and  missiles. 

•  AN/BYG-1  replaces  central  processors  with  commercial 
off-the-shelf  (COTS)  computer  technology.  The  Navy  installs 
improvements  to  the  system  via  an  incremental  development 
program.  The  program  includes  the  following: 

-  A  combat  control  system  for  the  Virginia  class  submarine 

-  A  replacement  combat  control  system  back-fit  into 
Los  Angeles,  Ohio,  and  Seawolf  class  submarines 

-  Biannual  software  upgrades  (called  Advanced  Processor 
Builds  (APBs))  and  hardware  upgrades  (called  Technology 
Insertions  (TIs)).  While  using  the  same  process  and 
nomenclature,  these  APBs  and  TIs  are  distinct  from  those 
used  in  the  Acoustic  Rapid  COTS  Insertion  (A-RCI) 
program. 

•  The  Navy  intends  improvements  to  provide  expanded 
capabilities  for  Anti-Submarine  (ASW)  and  Anti- Surface 


Warfare  (ASUW),  high-density  contact  management,  and  the 
targeting  and  control  of  submarine  weapons. 

•  The  Navy  is  also  developing  AN/BYG-1  for  use  on  the  Royal 
Australian  Navy  Collins  class  diesel  electric  submarines. 

Mission 

Submarine  crews  equipped  with  the  AN/BYG-1  combat  control 

system  are  able  to  complete  the  following  submarine  force 

missions: 

•  Analyze  submarine  sensor  contact  information  to  track 
submarine  and  surface  vessels  in  open  ocean  and  littoral  sea 
environments 

•  Employ  heavyweight  torpedoes  against  submarine  and  surface 
ship  targets 

•  Receive  strike  warfare  tasking,  plan  strike  missions,  and 
employ  Tomahawk  land-attack  cruise  missiles 

•  Receive  and  synthesize  all  organic  sensor  data  and  external 
tactical  intelligence  to  produce  an  integrated  tactical  picture 

Major  Contractors 

•  General  Dynamics  Advanced  Information  Systems  -  Fairfax, 
Virginia 

•  General  Dynamics  Advanced  Information  Systems  -  Pittsfield, 
Massachusetts 


Activity 

•  The  Navy  completed  FOT&E  of  AN/BYG-1  APB-09  in  early 
FY12  in  accordance  with  a  DOT&E-approved  test  plan.  The 
Navy  combined  testing  with  the  AN/BQQ-10  A-RCI  Sonar 
System,  the  Virginia  class  submarine,  and  the  Mk  48  Mod  6 
Advanced  Common  Torpedo  (ACOT)  and  Mk  48  Advanced 
Capability  (ADCAP)  Mod  7  Common  Broadband  Advanced 
Sonar  System  (CBASS)  programs.  Coordinating  these  tests 
provided  testing  efficiencies  and  enabled  an  end-to-end 
evaluation  of  mission  performance. 


•  DOT&E  issued  a  classified  combined  operational  test  report  in 
November  2012  that  evaluates  the  effectiveness  and  suitability 
of  the  AN/BYG-1  APB-09  system. 

•  The  Navy  began  drafting  a  Test  and  Evaluation  Master  Plan 
(TEMP)  for  the  APB- 11  and  APB- 13  variants  of  AN/BYG-1, 
and  expects  to  issue  it  by  early  FY13.  Operational  testing 
of  the  APB-1 1  variant  of  AN/BYG-1  is  expected  to  begin  in 
FY13. 
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Assessment 

•  The  Navy’s  schedule-driven  process  prevents  operational  test 
results  from  directly  supporting  development  of  the  follow-on 
APBs.  The  Navy  completed  operational  testing  of  the 
AN/BYG-1  APB-09  system  in  2011.  Due  to  the  combination 
of  late  completion  of  testing  and  the  Navy’s  practice  of  issuing 
an  updated  software  and  hardware  version  every  two  years, 
data  from  APB-09  operational  testing  have  not  been  included 
in  the  development  of  APB-11,  which  is  nearing  completion. 

•  The  DOT&E  classified  combined  report  to  Congress  for  the 
Virginia  class  submarine,  A-RCI  APB-09,  and  AN/BYG-1 
APB-09  systems  concluded  the  following  regarding 
AN/BYG-1  test  adequacy  and  system  performance: 

-  Given  the  data  available,  no  evidence  exists  to  change  the 
conclusions  about  mission  performance  from  previous 
reports  on  AN/BYG-1  (not  improved  or  degraded). 
Specifically, 

■  APB-09  is  not  effective  in  supporting  operator  situational 
awareness  and  contact  management  in  areas  of 
high-contact  density. 

■  APB-09  did  not  meet  the  Navy’s  requirements  for  target 
localization;  however,  the  targeting  solutions  were  often 
sufficient  for  a  trained  crew  to  provide  the  torpedo 

an  opportunity  to  detect  the  target.  Nevertheless,  the 
APB-09  variant  remains  not  effective  in  ASW  scenarios. 

■  DOT&E’s  overall  assessment  of  Information  Assurance 
was  unchanged  from  the  APB-07  variant  (not  effective), 
although  the  new  software  represents  an  improvement  in 
Information  Assurance  over  previous  systems. 

■  APB-09  is  operationally  suitable  and  continues  to  exhibit 
excellent  reliability  and  availability;  however,  the  Navy 
needs  to  improve  APB  training. 

-  Testing  to  examine  missions  involving  ASW-Attack, 
situational  awareness,  and  contact  management  in  areas 
of  high-contact  density  was  adequate  for  the  system 
tested  but  not  adequate  for  the  software  version  fielded 
on  Virginia  class  submarines.  The  AN/BQQ- 10  A-RCI 
processing  for  the  WAA  suffered  from  system  development 
problems,  which  led  to  poor  performance  during  the 
operational  testing.  AN/BYG-l’s  contribution  to  the  ASW 
mission  performance  and  submarine  mission  operations 

in  areas  of  high-contact  density  was  hindered  due  to  these 
problems.  The  Navy  investigated  the  WAA  issues  after 
operational  testing  was  completed  and  subsequently  issued 
software  fixes  intended  to  correct  the  problems.  The  Navy 
conducted  some  limited  developmental  testing  to  confirm 
functionality;  however,  the  Navy  has  not  completed 


operational  testing  to  evaluate  the  updated  WAA  software 
effects  on  BYG-1  performance.  DOT&E  assesses  that 
the  completed  operational  testing  allows  for  an  adequate 
assessment  of  the  performance  on  Los  Angeles  class  and 
Ohio  class  submarines  that  do  not  have  a  WAA. 

-  The  new  BYG-1  route  planning  tools  were  unable  to  be 
fully  evaluated  due  to  the  lack  of  required  databases  and 
crew  training. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
made  progress  in  addressing  four  of  the  seven  outstanding 
recommendations  contained  in  previous  annual  reports 
and  test  reports.  The  remaining  significant  unclassified 
recommendations  are: 

1 .  Improve  the  detection  and  localization  performance 
for  submarines  operating  in  high-density  surface  ship 
environments.  Consider  investing  in  automation  that 
will  assist  the  operator  in  processing  the  large  amount  of 
constantly  changing  contact  data  and  determining  which 
contacts  pose  an  immediate  collision  or  counter-detection 
threat. 

2.  Improve  operator  training  such  that  operators  understand 
and  effectively  employ  new  APB  functionality  when 
fielded. 

3.  Conduct  future  ASW  testing  against  high-end  targets 
simulating  modern  threat  diesel-electric  submarines  (SSK) 
to  enable  a  more  complete  assessment  of  A-RCI  and  BYG-1 
performance.  Since  acoustic  sensors  and  environmental 
conditions  effect  the  system’s  performance,  testing  should 
be  conducted  using  the  Navy’s  tactical  sensor  combinations 
and  in  a  variety  of  threat-like  environments. 

•  FY12  Recommendations. 

1.  DOT&E  published  a  classified  APB-09  FOT&E  report  in 
November  2012.  That  report  identifies  21  recommendations 
that  the  Navy  should  address  for  the  A-RCI  sonar  system 
and  the  AN/BYG-1  combat  control  systems.  In  particular, 
the  Navy  should  re-evaluate  operational  effectiveness  on  a 
submarine  with  a  repaired  WAA. 

2.  The  Navy  should  consolidate  the  A-RCI  and  AN/BYG-1 
TEMPs  and  test  plans  into  an  Undersea  Enterprise  Capstone 
document  to  permit  efficiencies  in  testing. 

3.  The  Navy  should  improve  its  developmental  testing 
processes  and  metrics  used  to  determine  if  a  system 
potentially  improves  effectiveness  and  suitability  and  to 
ensure  that  the  system  is  ready  for  operational  testing  and 
subsequent  fielding. 
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Common  Aviation  Command  and  Control  System 

(CAC2S) 


Executive  Summary 

•  In  September  2011,  DOT&E  assessed  that  the  Common 
Aviation  Command  and  Control  System  (CAC2S)  Phase  1 
IOT&E  was  not  adequate  due  to  test  venue  limitations 
and  insufficient  data  collection.  As  a  result,  DOT&E 
could  not  determine  CAC2S  operational  effectiveness  and 
suitability. 

•  In  September  2011,  the  Assistant  Secretary  of  the  Navy, 
Research,  Development,  and  Acquisition,  as  the  Milestone 
Decision  Authority  (MDA),  led  the  F ull  Deployment 
Decision  Review.  The  MDA  subsequently  approved 
Phase  1  full  deployment  of  CAC2S  and  directed  the 
CAC2S  program  manager  to  conduct  additional  testing 

to  address  recommendations  by  DOT&E  and  the  Marine 
Corps  Test  and  Evaluation  Activity  (MCOTEA)  based  on 
deficiencies  identified  during  the  IOT&E. 

•  In  lune  2012,  the  Marine  Corps  completed  a  Limited  User 
Evaluation  (LUE)  for  CAC2S  Increment  1,  Phase  1  that 
satisfactorily  addressed  previously  identified  key  DOT&E 
and  MCOTEA  deficiencies. 

•  Based  on  data  collected  during  the  LUE  combined 
with  that  from  the  IOT&E,  DOT&E  assessed  CAC2S 
Increment  1,  Phase  1  as  operationally  effective  and 
operationally  suitable  in  an  October  2012  Major 
Automated  Information  System  report  for  CAC2S 
Increment  1,  Phase  1. 

System 

•  CAC2S  is  designed  to  provide  Marine  Corps  operators 
with  the  ability  to  share  mission-critical  voice,  video, 
sensor,  and  command  and  control  data  and  information  in 
order  to  integrate  aviation  and  ground  combat  planning 
and  operations  in  support  of  the  Marine  Air-Ground  Task 
Force  (MAGTF). 

•  CAC2S  consists  of  tactical  shelters,  software,  and 
common  hardware.  The  hardware  components  are 
expeditionary,  common,  modular,  and  scalable,  and  may 
be  freestanding,  mounted  in  transit  cases,  or  rack-mounted 
in  shelters  and/or  general  purpose  tents  that  are  transported 
by  organic  tactical  mobility  assets. 

•  CAC2S  Increment  1  is  comprised  of  three  functional 
subsystems  to  be  delivered  in  two  phases. 

-  Phase  1: 

■  Processing  and  Display  Subsystem  (PDS)  -  Provides 
the  operational  command  post  and  functionality  to 
support  mission  planning,  decision  making,  and 
execution  tools  for  all  aspects  of  Marine  Aviation. 

■  Communication  Subsystem  (CS)  -  Provides  the 
capability  to  interface  with  internal  and  external 


CAC2S  Increment  I,  Phase  I 


communication  assets  and  the  means  to  control  their 
operation. 

-  Phase  2: 

■  Sensor  Data  Subsystem  (SDS)  -  Provides  an 
open-architecture  interface  capable  of  integrating 
emerging  active  and  passive  sensor  technology  for 
organic  and  non-organic  sensors  of  the  Marine  Air 
Command  and  Control  System  (MACCS). 

•  CAC2S  Phase  1  includes  the  PDS  and  CS  to  establish  the 
baseline  Direct  Air  Support  Center  (DASC)  for  the  Marine 
Air  Support  Squadron  and  limited  Tactical  Air  Operations 
Center  (TAOC)  mission  functionality  for  the  Marine  Air 
Control  Squadron.  During  Phase  2,  SDS  is  intended  to 
enhance  the  CAC2S  and  meet  remaining  MACCS  aviation 
battle  management  command  and  control  requirements  to 
include  fusion  of  real-time  data. 

Mission 

•  The  MAGTF  Commander  will  employ  CAC2S  to 
integrate  Marine  Corps  aviation  into  joint  and  combined 
air/ground  operations  in  support  of  Operational  Maneuver 
from  the  Sea,  Sustained  Operations  Ashore,  and  other 
expeditionary  operations.  The  CAC2S  will  support  the 
MAGTF  command  and  control  concept  and  will  provide 
an  expeditionary  and  common  joint  air  command  and 
control  capability. 

•  The  MAGTF  Commander  will  execute  command  and 
control  of  assigned  assets  afloat  and  ashore  in  a  joint, 
allied,  or  coalition  operational  environments  by  using 
CAC2S  capabilities  to: 

-  Display  a  common,  real  and  near  real-time  integrated 
tactical  picture  with  the  timeliness  and  accuracy 
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necessary  to  facilitate  the  control  of  friendly  assets  and 
the  engagement  of  threat  aircraft  and  missiles 
Access  theater  and  national  intelligence  sources  from  a 
single,  multi-function  command  and  control  node 
Standardize  Air  Tasking  Order  (ATO)  and  Airspace 
Control  Order  generation,  parsing,  interchange, 
and  dissemination  throughout  the  MAGTF  and 
theater  forces  by  using  the  joint  standard  for  ATO 
interoperability 


Major  Contractors 

•  Phase  1 

-  Government  Integrator:  Naval  Surface  Warfare 
Center  -  Crane,  Indiana 

-  Component  Contractor:  Raytheon-Solipsys  -  Fulton, 
Maryland 

-  Component  Contractor:  General  Dynamics  -  Scottsdale, 
Arizona 

•  Phase  2:  General  Dynamics  -  Scottsdale,  Arizona 


Activity 

•  In  September  2011,  DOT&E  assessed  that  the  CAC2S  Phase  1 
IOT&E  was  not  adequate  due  to  test  venue  limitations  and 
insufficient  data  collection.  As  a  result,  DOT&E  could  not 
determine  CAC2S  operational  effectiveness  and  suitability. 

•  In  September  2011,  the  Assistant  Secretary  of  the  Navy, 
Research,  Development,  and  Acquisition,  as  the  MDA,  led  the 
Full  Deployment  Decision  Review.  The  MDA  subsequently 
approved  Phase  1  full  deployment  of  CAC2S  and  directed 
the  CAC2S  program  manager  to  conduct  additional  testing 

to  address  test  inadequacies  and  deficiencies  identified  by 
DOT&E  and  MCOTEA  during  the  IOT&E. 

•  The  Marine  Corps  conducted  an  LUE  in  accordance  with  a 
DOT&E-approved  test  plan  to  address  prior  limitations  in 
test  adequacy  and  to  test  corrections  for  CAC2S  deficiencies 
identified  during  the  IOT&E.  The  test  was  divided  into  two 
phases.  The  Marine  Corps  conducted  Phase  1  testing  during 
the  Weapons  and  Tactics  Instructor’s  Course  (WTI)  exercise  at 
Marine  Corps  Air  Station  Yuma,  Arizona,  in  April  2012.  The 
Marine  Corps  conducted  Phase  2  testing  at  the  Marine  Corps 
Tactical  Systems  Support  Activity  in  June  2012. 

•  The  Marine  Corps  began  test  planning  for  Phase  2  of  CAC2S 
in  FY12.  IOT&E  is  scheduled  for  FY15. 

Assessment 

•  The  primary  objectives  of  LUE  Phase  1  testing  were  to  assess 
the  TAOC  mission  in  an  operationally  realistic  environment 
and  CAC2S’s  ability  to  support  continuous  operations  during 
DASC  displacement.  The  primary  objectives  of  LUE  Phase  2 
testing  were  to  assess  accuracy  and  timeliness  of  the  Common 
Tactical  Picture,  Joint  Range  Extension  Application  Protocol 
A/B  connectivity,  Global  Command  and  Control  System 
integration.  Advanced  Field  Artillery  Data  System  integration, 
and  maximum  track  processing  capability.  Data  collected 
during  both  phases  of  the  LUE  were  used  to  assess  system 
reliability,  availability,  and  maintainability. 

•  The  LUE  satisfactorily  addressed  key  deficiencies  previously 
identified,  and  there  were  no  major  limitations  to  testing.  The 
minor  limitations  of  not  having  the  Service-level  preventative 
maintenance  plan  established,  as  well  as  not  having  a  logistics 
spares  and  delivery  structure  in  place,  did  not  hinder  the 
overall  assessment.  The  WTI  exercise,  during  which  the  LUE 
was  conducted,  did  allow  sufficient  data  collection  to  resolve 
TAOC  effectiveness  operations.  The  combination  of  both 


IOT&E  and  LUE  testing  provided  adequate  data  to  assess  the 
operational  effectiveness  and  suitability  of  CAC2S. 

•  The  CAC2S  is  operationally  effective  in  its  primary  mission  to 
support  the  DASC  and  TAOC.  DASC  operators  were  able  to 
use  the  CAC2S  to  support  direct  air  support  missions  during 
both  the  IOT&E  and  LUE.  The  CAC2S  Common  Tactical 
Picture  provided  DASC  operators  with  timely,  accurate,  and 
relevant  information.  Early  Warning  and  Control  TAOC 
operators  were  able  to  use  the  CAC2S  communications 
capabilities  to  support  airspace  surveillance,  air  direction 

and  control,  coordination  of  air  assets,  and  weapons  systems 
integration  during  the  LUE. 

•  The  CAC2S  is  operationally  suitable.  The  combination  of 
data  from  both  the  IOT &E  and  LUE  are  adequate  to  assess 
reliability,  availability,  and  maintainability.  DASC  and  TAOC 
operators  were  able  to  consistently  employ  the  system  in  the 
operationally  representative  environment  provided  during  WTI 
operations.  The  reliability  of  completing  a  24-hour  mission 
was  88  percent,  and  the  estimated  Mean  Time  Between 
Operational  Mission  Failure  was  192  hours.  The  operational 
availability  was  calculated  as  97.9  percent. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Marine  Corps 
addressed  all  previous  FY10  and  FY11  recommendations. 

•  FY12  Recommendations.  The  Marine  Corps  should: 

1.  Ensure  that  future  test  venues  for  Phase  2  IOT&E  provide 
for  a  balanced  use  of  air  and  ground  combat  forces  along 
with  a  24-hour  operational  window  to  ensure  adequate 
system  operating  hours  for  assessment  of  system  reliability. 
In  order  to  reduce  risk,  an  operational  assessment  should 
be  conducted  prior  to  IOT&E  that  assesses  functionality, 
integration,  and  employment  for  all  MACCS  users. 

2.  Develop  and  implement  a  data  collection  plan  to  aid  in 
assessment  of  reliability,  availability,  and  maintainability 
requirements  of  currently  fielded  systems  and  use  the  data 
to  supplement  the  evaluations  of  Phase  2. 

3.  Establish  requirements  for  and  develop  a  Service-level 
preventative  maintenance  plan. 

4.  Conduct  a  supportability  assessment  to  assess  the 
availability  of  logistic  support  and  spares  for  Phase  2. 
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CV-22  Osprey 


Executive  Summary 

•  Air  Force  Special  Operations  Command  (AFSOC)  conducted 
recent  operational  testing  of  upgrades  to  the  Directional 
Infrared  Countermeasure  (D1RCM)  and  Suite  of  Integrated 
Radio  Frequency  Countermeasures  (SIRFC)  systems  that 
demonstrated  improvements  in  countermeasure  effectiveness. 

•  AFSOC  completed  operational  assessments  of  ramp-mounted 
machine  guns  and  flight  control  software  in  FY12. 

•  AFSOC  plans  to  evaluate  improvements  to  the  icing  protection 
and  communications  systems  in  FY13. 

System 

•  There  are  two  variants  of  the  V-22:  the  Marine  Corps 
MV-22  and  the  Air  Force/U.S.  Special  Operations  Command 
(USSOCOM)  CV-22.  The  air  vehicles  for  Air  Force  and 
Marine  Corps  missions  are  nearly  identical,  with  common 
subsystems  and  military  components  sustainable  by  each 
Service’s  logistics  system. 

•  The  CV-22  is  the  replacement  for  aging  Special  Forces  MF1-53 
helicopters.  It  is  a  tilt-rotor  aircraft  capable  of  conventional 
fixed-wing  flight  and  vertical  take-off  and  landing  over  the 
range  of  Special  Operations  missions. 

•  Its  speed  and  range  enable  the  ability  to  support  Special 
Operations  mission  demands  that  were  not  possible  with 
legacy  rotary-  or  fixed-wing  aircraft. 

•  The  CV-22  can  carry  18  combat-ready  Special  Operators 
538  nautical  miles  (nm)  and  return.  It  can  self-deploy  up  to 
2,100  nm  with  one  aerial  refueling. 

•  The  CV-22  will  augment  Air  Force  Special  Operations 
MC-130  aircraft.  It  has  terrain-following/terrain-avoidance 
radar,  an  advanced  multi-frequency  communication  suite,  and 
a  more  robust  electronic  defense  suite. 


•  Future  capabilities  will  include  engine  sub-assembly  upgrades, 
strategic  refueling  capability,  and  various  fixes  to  deficiencies 
identified  during  IOT&E. 

Mission 

Air  Force  squadrons  equipped  with  the  CV-22  will  provide  high 
speed,  long-range  insertion  and  extraction  of  Special  Operations 
Forces  to  and  from  high-threat  objectives. 

Major  Contractors 

Bell-Boeing  Joint  Venture: 

•  Bell  Helicopter  -  Amarillo,  Texas 

•  The  Boeing  Company  -  Ridley  Township,  Pennsylvania 


Activity 

•  To  address  deficiencies  with  electronic  defense  systems 
discovered  during  the  2008  CV-22  IOT&E,  AFSOC 
tested  upgraded  hardware  and  software  of  the  D1RCM 
(April  through  June  2010)  and  SIRFC  (April  through 
July  2012). 

•  The  CV-22  program  relocated  communication  antennas  to 
address  limited  operating  range  and  inadequate  reliability  with 
the  CV-22  radios  demonstrated  during  the  2008  IOT&E. 

•  AFSOC  conducted  operational  assessments  of  the  GAU-18 
and  GAU-21  .50  caliber  Upgraded  Ramp-Mounted  Weapon 
System  in  March  through  June  2012.  AFSOC  had  modified 
these  machine  guns  to  improve  weapons  reliability  following 
brownout  landings. 

•  AFSOC  evaluated  the  Tactical  Software  Suite  10.3.01  in 
March  through  April  2012.  The  major  feature  of  this  software 


version  was  the  addition  of  the  capability  to  perform  coupled 
or  uncoupled  tactical  approaches  (during  a  coupled  approach, 
flight  crew  utilize  autopilot  to  fly  the  aircraft;  during  an 
uncoupled  approach,  the  flight  crew  fly  the  aircraft  manually). 

Assessment 

•  DOT&E  evaluated  the  D1RCM  system  during  the  CV-22 
IOT&E  in  2009  and  reported  significant  reliability  and 
performance  shortfalls.  Since  IOT&E,  AFSOC  upgraded 
the  CV-22  D1RCM  with  a  new  system  processor,  updated 
software,  and  Guardian  Laser  Transmitter  Assembly  designed 
to  reduce  system  weight  and  to  improve  system  reliability. 
AFSOC  tested  the  hardware  and  software  upgrades  to 
the  CV-22  D1RCM  and  the  upgrades  meet  performance 
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requirements.  IOT&E  reliability  failures  have  not  resurfaced 
during  testing  or  fielding. 

•  DOT&E  evaluated  the  SIRFC  system  during  the  IOT&E 
and  reported  substantial  significant  shortfalls  in  providing 
threat  situational  awareness  and  limited  countermeasure 
effectiveness.  DOT&E  also  reported  a  high  rate  of  reliability 
failures  during  IOT&E.  Since  IOT&E,  AFSOC  upgraded 
the  CV-22  SIRFC  with  new  high-power  transmitters, 
cabling,  radio-frequency  switches,  antennas,  and  Block  7 
Operational  Flight  Software  with  over  140  changes  to  correct 
system  deficiencies;  nonetheless,  some  deficiencies  remain 
outstanding.  Analysis  of  the  SIRFC  test  data  from  FY12 
testing  is  ongoing  and  will  be  reported  in  2013. 

•  The  GAU-18  .50  caliber  Upgraded  Ramp-Mounted  Weapon 
System  is  not  effective  or  suitable  for  fielding.  Dust  covers 
proposed  for  the  GAU-18  did  not  protect  the  weapon  from 
dust  intrusion  and  resulted  in  numerous  broken  extractor  pins. 

•  In  testing,  the  GAU-21  demonstrated  potential  for  reliable 
performance  following  exposure  to  brownout  conditions. 
Repeated  brownout  landings  did  not  adversely  affect  the  rate 
of  fire. 

•  The  Tactical  Software  Suite  10.3.01  enabled  pilots  to  conduct 
coupled  and  uncoupled  tactical  approaches.  This  enhancement 
reduces  pilot  workload  when  coupled  approaches  are  feasible. 

•  While  the  program  has  made  progress  in  improving  CV-22 
radio  effectiveness  and  reliability,  further  testing  and 


assessment  is  needed.  Additionally,  deficiencies  with  the  Icing 
Protection  System  remain.  Improvement  and  further  testing  is 
needed  in  both  areas. 

•  No  development  or  testing  has  been  accomplished  to  address 
aerial  refueling  from  strategic  tankers. 

•  No  additional  flight  testing  or  engineering  analysis  have 
been  performed,  indicating  a  change  would  be  appropriate  to 
DOT&E ’s  September  2005  assessment  that  the  V-22  cannot 
perform  autorotation  to  a  survivable  landing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program 
addressed  all  but  one  of  the  previous  recommendations.  The 
recommendation  regarding  development  of  battle  damage 
repair  procedures  and  fire  suppression  systems  for  the  aircraft 
dry  bays  remains  valid. 

•  FY12  Recommendations. 

1.  The  program  should  address  deficiencies  with  the 
multi-mission  advanced  tactical  terminal  and  the  strategic 
refueling  capability  as  documented  in  IOT&E  and  then 
operationally  test  the  fixes. 

2.  AFSOC  should  proceed  with  plans  to  evaluate 
communications  effectiveness  and  reliability  in  FY13. 

3.  AFSOC  should  proceed  with  plans  to  evaluate  reliability 
fixes  to  the  Icing  Protection  System  under  operationally 
representative  icing  conditions  in  FY13. 
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CVN-78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 


Executive  Summary 

•  The  current  Test  and  Evaluation  Master  Plan  (TEMP)  does  not 
adequately  address  integrated  platform-level  developmental 
testing,  significantly  raising  the  likelihood  that  platform-level 
test  problems  will  be  discovered  during  IOT&E.  The  Program 
Office  has  begun  to  address  the  problem  and  has  refined 

the  post-delivery  schedule.  However,  the  details  as  to  the 
extent  of  any  additional  integrated  platform-level  CVN-78 
developmental  tests  are  unclear. 

•  The  Navy  began  CVN-78  construction  in  2008  and  plans  to 
deliver  the  ship  in  September  2015.  Current  progress  supports 
this  plan;  however,  the  Electro-Magnetic  Aircraft  Launch 
System  (EMALS),  Advanced  Arresting  Gear  (AAG),  Dual 
Band  Radar  (DBR),  and  Integrated  Warfare  Systems  will 
continue  to  drive  the  timeline  for  successful  delivery  of  the 
ship. 

•  The  Navy  continues  to  work  on  integration  deficiencies  related 
to  the  F-35  Joint  Strike  Fighter  (JSF)  and  its  fleet  of  aircraft 
carriers,  including  CVN-78. 

•  Although  CVN-78  will  include  a  new  heavy  underway 
replenishment  (UNREP)  system  that  will  allow  transfer  of 
12,000-pound  cargos,  the  Navy’s  plan  to  install  heavy  UNREP 
systems  on  resupply  ships  has  slipped  eight  years. 

•  DOT&E  rescinded  approval  of  the  alternative  LFT&E 
Management  Plan  pertaining  to  the  Gerald  R.  Ford  (CVN-78) 
class  carrier  program  because  the  Navy  unilaterally  decided 
to  modify  the  previously  approved  LFT&E  plan.  The  Navy 
wanted  to  limit  the  scope  of  the  Total  Ship  Survivability  Trial 
(TSST)  on  CVN-78  to  conform  to  the  Navy  budget,  and  to 
defer  the  Full-Ship  Shock  Trial  (FSST)  to  CVN-79,  a  change 
to  the  previously  approved  LFT&E  Management  Plan  with 
which  DOT&E  does  not  concur. 

System 

•  The  CVN-78  Gerald  R.  Ford  class  nuclear  aircraft  carrier 
program  is  a  new  class  of  nuclear  powered  aircraft  carriers  that 
replaces  the  previous  CVN-21  program  designation.  It  has 
the  same  hull  form  as  the  CVN-68  Nimitz  class,  but  many  ship 
systems,  including  the  nuclear  plant  and  the  flight  deck,  are 
new. 

•  The  newly  designed  nuclear  power  plant  is  intended  to  reduce 
reactor  department  manning  by  50  percent  and  produce 
significantly  more  electricity  when  compared  to  a  current 
CVN-68  class  ship. 

•  The  CVN-78  will  incorporate  electromagnetic  catapults 
(instead  of  steam-powered),  and  have  a  smaller  island  with 
a  DBR  (a  phased-array  radar  which  replaces/combines  five 
legacy  radars  used  on  current  aircraft  carriers). 

•  The  Navy  intends  for  the  Integrated  Warfare  System  to 
be  adaptable  to  technology  upgrades  and  varied  missions 


throughout  the  ship’s  projected  operating  life  including 
increased  self-defense  capabilities  when  compared  to  current 
aircraft  carriers. 

•  The  Navy  redesigned  weapons  stowage,  handling  spaces,  and 
elevators  to  reduce  manning,  increase  safety,  and  increase 
throughput  of  weapons. 

•  The  Navy  designed  CVN-78  to  increase  the  sortie  generation 
capability  of  embarked  aircraft  to  160  sorties  per  day  (12-hour 
fly  day)  and  to  surge  to  270  sorties  per  day  (24-hour  fly  day) 
as  compared  to  the  CVN-68  Nimitz  class  sortie  generation  rate 
demonstration  of  120  sorties  per  day/240  sorties  for  24-hour 
surge. 

•  The  Consolidated  Afloat  Networks  and  Enterprise  Services 
(CANES)  program  replaces  five  shipboard  legacy  network 
programs  to  provide  a  common  computing  environment  for 
command,  control,  intelligence,  and  logistics. 

•  The  Navy  plans  to  declare  CVN-78  Initial  Operational 
Capability  in  FY17  with  Full  Operational  Capability  in  FY18 
after  the  Milestone  C  decision. 

Mission 

Carrier  Strike  Group  Commanders  will  use  the  CVN-78  to: 

•  Conduct  power  projection  and  strike  warfare  missions  using 
embarked  aircraft 

•  Provide  force  protection  of  friendly  units 

•  Provide  a  sea  base  as  both  a  command  and  control  platform 
and  an  air-capable  unit 

Major  Contractor 

Huntington  Ingalls  Industries,  Newport  News 

Shipbuilding  -  Newport  News,  Virginia 
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Activity 

Test  Planning 

•  The  Navy  continues  to  develop  the  CVN-78  Sortie 
Generation  Rate  (SGR)  (number  of  aircraft  sorties  per  day) 
test  concept.  Discussions  have  focused  on  the  specific 
details  of  live  testing  days  (e.g.,  which  test  ranges  to  use, 
how  many  aircraft,  which  weapons).  The  ship’s  SGR 
requirement  is  based  on  a  30-plus-day  wartime  scenario. 
DOT&E  concurs  with  the  proposed  6  consecutive  12-hour 
fly  days  followed  by  2  consecutive  24-hour  fly  days. 

This  live  testing  will  be  supplemented  with  modeling  and 
simulation  from  the  Virtual  Carrier  model  to  extrapolate 
results  to  the  30-plus-day  SGR  requirement. 

•  DOT&E  approved  an  operational  assessment  that  began 
in  September  2012,  to  assess  the  progress  of  the  CVN-78 
build  and  its  ability  to  successfully  undergo  operational 
test  and  evaluation  in  the  future.  The  assessment  is  being 
led  by  U.S.  Navy’s  Commander,  Operational  Test  and 
Evaluation  Force  and  is  a  five-month  series  of  site  visits 
and  evaluations  by  fleet  experienced  operators  who  will 
perform  a  detailed  analysis  of  the  carrier  and  all  its  major 
subcomponents.  This  review  is  expected  to  be  completed 
in  2QFY13  and  will  be  used  to  inform  the  Defense 
Acquisition  Board  decision  regarding  future  procurement  of 
CVN-79. 

•  The  CVN-78  Gerald  R.  Ford  class  carrier  program 
Office  is  revising  the  TEMP  in  an  effort  to  align  planned 
developmental  tests  with  corresponding  operational  test 
phases  and  to  identify  needed  platform-level  developmental 
testing.  As  part  of  this  process,  the  Program  Office  recently 
released  a  Post  Delivery  Test  and  Trials  schedule. 

EMALS 

•  The  EMALS  system  functional  design  test  site  at  Joint  Base 
McGuire-Dix-Lakehurst,  New  Jersey,  continues  to  test  the 
new  electromagnetic  catapult  system. 

•  The  Navy  completed  Phase  I  of  Aircraft  Compatibility 
Testing  and  installed  a  re-designed  armature.  Additionally, 
the  system  was  reconfigured  to  enable  testing  of  simulated 
shared  energy  storage  and  simulated  shared  power 
conversion  to  provide  an  early  examination  of  multiple 
catapults  on  a  carrier. 

AAG 

•  The  Navy  is  testing  the  AAG  on  a  jet  car  track  at  Joint 
Base  McGuire-Dix-Lakehurst,  New  Jersey.  Earlier  testing 
prompted  design  changes  for  the  system’s  Water  Twisters, 
Cable  Shock  Absorbers,  Inverters,  and  Purchase  Cable 
Drum  frames.  In  August  2012,  the  program  completed  two 
test  milestones  that  validated  system  performance  in  the 
uppermost  regions  of  the  system  performance  envelope. 
AAG  performance  testing  began  in  September  2012. 

•  As  of  October  2012,  about  one  third  of  AAG  hardware 
items  have  been  delivered  to  the  shipyard. 

JSF 

•  The  Navy  completed  land-based  JSF  testing  associated 
with  the  Jet  Blast  Deflector  (JBD).  The  JBD  is  designed  to 
deflect  engine  exhaust  during  catapult  launches. 


CANES 

•  The  Navy  conducted  developmental  testing  on  the 
unit-level  CANES  configuration  used  on  Aegis  destroyers 
in  the  lab  from  July  11  -  24,  2012.  The  Navy  has 
scheduled  developmental  and  follow-on  testing  of  the 
force-level  CANES  configuration  used  on  the  Nimitz  and 
Gerald  R.  Ford  classes  for  the  4QFY14. 

DBR 

•  The  Navy  is  reactivating  the  Engineering  Development 
Model  of  the  Volume  Search  Radar  portion  of  the  DBR 
at  the  surface  Combat  System  Center  at  Wallops  Island, 
Virginia.  In  addition,  the  Navy  is  installing  a  production 
Multi-Functional  Radar  component  of  DBR  to  establish 
capability  by  January  2013  to  support  CVN-78  combat 
system  integration  and  test. 

•  The  Navy  plans  to  conduct  DBR  testing  at  Wallops  Island, 
Virginia,  to  verify  the  radar  will  meet  CVN-78’s  Air  Traffic 
Control  requirements  in  January  2013.  The  Navy  will  also 
begin  CVN-78  combat  system  integration  testing  with  DBR 
in  April  2013. 

LFT&E 

•  DOT&E  rescinded  approval  of  the  alternative  LFT&E 
Management  Plan  pertaining  to  the  Gerald  R.  Ford  class 
carrier  program  because  the  Navy  unilaterally  decided  to 
modify  the  previously  approved  LFT&E  plan.  The  Navy 
informed  DOT&E  of  its  intent  to  limit  the  scope  of  the 
TSST  on  CVN-78  to  conform  to  the  Navy  budget.  While 
progress  has  been  made  toward  reaching  consensus  on 
the  scope  of  the  TSST,  the  budget  has  not  been  adjusted 
accordingly.  The  Navy  also  deferred  the  FSST  to  CVN-79, 
a  change  to  the  previously  approved  LFT&E  Management 
Plan  with  which  DOT&E  does  not  concur.  Though  the 
change  is  motivated  by  the  desire  to  reduce  the  gap  in 
available  carriers  caused  by  the  retirement  of  the  USS 
Enterprise,  the  delay  due  to  the  FSST  is  minimal,  and  only 
a  small  portion  of  the  already  substantial  delay  caused  by 
other  factors.  The  benefit  of  having  test  data  to  affect  the 
design  of  future  carriers  in  the  class  outweighs  the  delay. 

Assessment 

Test  Planning 

•  The  current  state  of  the  Virtual  Carrier  model  does  not 
fully  provide  for  an  accurate  accounting  of  SGR  due  to  a 
lack  of  fidelity  regarding  manning  and  equipment/aircraft 
availability.  Spiral  development  of  the  Virtual  Carrier 
model  continues  in  order  to  ensure  that  the  required  fidelity 
will  be  available  to  support  the  SGR  assessment  during 
IOT&E. 

•  The  current  TEMP  does  not  adequately  address  whole 
platform-level  developmental  testing.  The  Program  Office 
has  begun  to  address  the  problem  and  has  refined  the  Post 
Delivery  Test  and  Trials  schedule.  The  details  are  unclear 
on  the  extent  of  any  additional  integrated  platform-level 
CVN-78  developmental  tests.  Lack  of  platform-level 
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developmental  testing  significantly  raises  the  likelihood  of 
the  discovery  of  platform-level  problems  during  IOT&E. 

•  The  Navy  plans  to  deliver  CVN-78  in  September  2015. 
Current  progress  supports  this  plan;  however,  the  EMALS, 
AAG,  DBR,  and  Integrated  Warfare  Systems  remain  pacing 
items  for  successful  delivery  of  the  ship. 

EMALS 

•  DOT&E  holds  moderate  concern  regarding  the  performance 
risk  generated  by  the  inability  to  test  the  full  four-catapult 
electrical  distribution  system  prior  to  initial  trials  aboard 
ship.  This  concern  is  partially  mitigated  by  the  current 
phase  of  test  using  a  simulated  shared  electrical  storage  and 
shared  power  conversion  at  the  EMALS  system  functional 
design  test  site  in  Joint  Base  McGuire-Dix-Lakehurst,  New 
Jersey. 

AAG 

•  Significant  redesign  of  multiple  components  has  slowed 
development  of  AAG  development.  The  program  will 
begin  performance  testing  in  FY13. 

JSF 

•  JBD  testing  identified  no  deficiencies  for  catapult  launch 
operations  of  JSF  at  military  rated  thrust.  The  tests  did, 
however,  determine  that  additional  JBD  side  panel  cooling 
(SPC)  and  other  adjustments  are  required  for  operations  at 
combat  rated  thrust,  i.e.,  with  afterburner.  The  existing  JBD 
panels  will  need  to  be  replaced  with  new  panels  with  SPC 
to  be  fully  JSF-compatible.  JBD  panels  with  SPC  are  form, 
fit,  and  function  replacements  and  will  be  installed  after 
CVN-78  delivers. 

-  JSF  data  flow  aboard  ship  via  the  Autonomic  Logistics 
Information  System  (ALIS)  is  critical  to  proper  F-35 
maintenance.  Currently,  the  ALIS  system  has  provided 
all  required  parametric  information  to  interface  properly 
with  CANES,  but  CANES  is  not  fully  developed  yet, 

as  the  contract  was  awarded  in  August  2012.  ALIS  is 
expected  to  undergo  Application  Integration  Process 
testing  in  FY13  to  ensure  proper  interface  with  CANES. 
DOT&E  will  be  able  to  better  assess  the  impact  on  JSF 
operations  aboard  CVN-78  after  the  test.  Currently,  data 
are  planned  to  be  exchanged  manually  until  ALIS  and 
CANES  properly  interface. 

-  In  2007,  the  Program  Office  identified  discrepancies 
with  the  integration  of  the  JSF’s  F135  engine  onto 
aircraft  carriers.  The  weight  of  the  FI  35  power  module, 
approximately  10,000  pounds,  exceeds  the  limit  of 
current  underway  replenishment  (UNREP)  systems. 
Although  CVN-78  will  include  a  heavy  UNREP  system 
that  will  allow  transfer  of  12,000  pounds,  supply  ships 
must  include  the  new  system  for  power  module  transfer 
to  occur.  The  Navy’s  plan  to  install  heavy  UNREP 
systems  on  resupply  ships  has  slipped  eight  years. 

•  Navy  Fleet  Force’s  JSF  “day-in-the-life”  analysis  identified 
a  significant  number  of  aircraft-ship  interface  deficiencies 
that  must  be  accomplished  by  the  Navy  in  post-delivery 
ship  modification.  They  include  the  following: 


-  JSF  battle  damage  assessment  and  non-traditional 
Intelligence,  Surveillance,  and  Reconnaissance 
information  captured  on  the  Portable  Memory  Device 
or  cockpit  video  recorder  cannot  be  shared  real-time 
with  the  Distributed  Common  Ground  System-Navy 
(DCGS-N).  This  prevents  assessment  by  shipboard 
intelligence  analysts  for  inclusion  in  mission  reports. 

-  Ships  are  unable  to  receive  and  display  Link  16  imagery; 
this  problem  is  not  unique  to  JSF.  The  Combatant 
Commanders  have  stated  a  need  to  display  imagery 

to  intelligence  analysts  and  operations  command  and 
control  nodes  to  enhance  engagement  decisions. 

-  Limited  shipboard  capabilities  exist  with  expeditionary 
Link  16.  The  Navy  is  considering  a  program  of 
record  to  distribute  imagery  to  analysts  and  maritime 
operations  command  and  control  nodes  (e.g.,  carriers 
and  amphibious  ships).  This  would  be  a  temporary 
workaround  for  the  DCGS-N  post-flight  data  gap. 

-  The  JSF  Prognostic  Health  Maintenance  (PHM) 
downlink  design  for  ships  is  not  mature.  The  uncertainty 
in  the  technical  characteristics  of  the  final  design 
means  that  there  are  potential  challenges  to  integrating 
PHM  into  current  shipboard  communications  suites 

and  networks.  These  challenges  include  unidentified 
Information  Assurance  considerations  and  unidentified 
waveform  hosting  and  interfacing. 

-  The  JSF  wheel  supplier’s  recent  rim  inspection 
requirement  may  force  a  significant  increase  in  shipboard 
tire  and  wheel  storage  requirements.  The  JSF  Program 
Office  is  currently  working  to  determine  the  effect  of 
this  deficiency  and  the  need  for  inspection  by  the  wheel 
supplier. 

DBR 

•  Previous  testing  of  Navy  combat  systems  similar  to 
CVN-78’s  revealed  numerous  integration  problems  that 
degrade  the  performance  of  the  combat  system.  The 
previous  results  emphasize  the  necessity  of  maintaining  a 
DBR/CVN-78  combat  system  asset  at  Wallops  Island.  The 
Navy  is  considering  long-term  plans  (i.e.,  beyond  FY15)  for 
testing  DBR  at  Wallops  Island,  Virginia,  but  it  is  not  clear 

if  a  Multi-Functional  Radar  and  funding  will  be  available. 
Such  plans  are  critical  to  delivering  a  fully  capable  combat 
system  and  ensuring  lifecycle  support  after  CVN-78 
delivery  in  2015. 

LFT&E 

•  While  the  Navy  has  made  substantial  effort  in  component 
and  surrogate  testing,  this  work  does  not  obviate  the  need 
to  conduct  the  FSST  to  gain  the  critical  empirical  data  that 
past  testing  has  repeatedly  demonstrated  are  required  to 
rigorously  evaluate  the  ship’s  ability  to  withstand  shock 
and  survive  in  combat.  Shock  Trials  conducted  on  both 
the  Nimitz  class  aircraft  carrier  and  the  San  Antonio  class 
Amphibious  Transport  Dock  demonstrated  the  need  for  and 
substantial  value  of  conducting  the  FSST.  Postponing  the 
FSST  until  CVN-79  would  cause  a  five-  to  seven-year  delay 
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in  obtaining  the  data  critical  to  evaluating  the  survivability 
of  the  CVN-78  and  would  preclude  timely  modification  of 
subsequent  ships  of  this  class  to  assure  their  survivability. 

•  The  Navy  proposes  delaying  the  shock  trial  by  five  to  seven 
years  because  of  the  approximately  four-  to  six-month 
delay  required  to  perform  the  FSST.  The  delay  is  not  a 
sufficient  reason  to  postpone  the  shock  trial,  since  the  shock 
trial  could  reveal  valuable  lessons,  including  previously 
unknown  vulnerabilities. 

•  DOT&E  has  requested  the  Navy  adequately  fund  and 
complete  the  actions  necessary  to  conduct  the  TSST 
on  the  CVN-78.  This  includes  updating  the  Damage 
Scenario  Based  Engineering  Analyses  (DSBEA)  from 
prior  Vulnerability  Assessment  Reports  and  enough  new 
DSBEAs,  including  machinery  spaces,  to  conduct  an 
adequately  scoped  TSST.  DOT&E  expects  this  will  require 
five  or  six  TSST  drills.  While  progress  has  been  made 
toward  reaching  consensus  on  the  scope  of  the  TSST,  there 
is  still  work  to  be  done,  mainly  to  include  the  machinery 
spaces,  and  the  budget  has  not  been  adjusted  to  adequately 
support  the  TSST. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  addressed 
one  of  eight  previous  recommendations  but  the  following 
seven  remain  valid: 


1 .  Adequately  test  and  address  integration  challenges  with 
JSF;  specifically  logistics  (storage  of  spare  parts  and 
engines,  transport  of  support  equipment  and  spares  to/from 
the  carrier),  changes  required  to  JBDs,  changes  (due  to 
heat  and  or  noise)  to  flight  deck  procedures,  and  AL1S 
integration. 

2.  Finalize  plans  that  address  CVN-78  integrated  warfare 
system  engineering  and  ship’s  self-defense  system 
discrepancies  prior  to  the  start  of  10T&E. 

3.  Continue  aggressive  EMALS  and  AAG  risk-reduction 
efforts  to  maximize  opportunity  for  successful  system 
design  and  test  completion  in  time  to  meet  required  in-yard 
dates  for  shipboard  installation  of  components. 

4.  Continue  development  of  a  realistic  model  for  determining 
CVN-78’s  SGR,  while  utilizing  realistic  assumptions 
regarding  equipment  availability,  manning,  and  weather 
conditions  for  use  in  the  IOT&E. 

5.  Provide  scheduling,  funding,  and  execution  plans  to 
DOT&E  for  the  live  SGR  test  event  during  the  IOT&E. 

6.  Continue  to  work  with  the  Navy’s  Bureau  of  Personnel  to 
ensure  adequate  depth  and  breadth  of  required  personnel 
to  ensure  that  the  100  percent  Navy  Enlisted  Classification 
fit/fill  manning  requirements  of  CVN-78  are  met. 

7.  Conduct  system-of-systems  developmental  testing  to 
preclude  discovery  of  deficiencies  during  IOT&E. 

•  FY12  Recommendations.  None. 
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Direct  Attack  Moving  Target  Capability  (DAMTC) 


Executive  Summary 

•  The  Direct  Attack  Moving  Target  Capability  (DAMTC)  is  a 
non-developmental  program  that  expands  the  employment 
envelope  of  the  original  Laser  Joint  Direct  Attack  Munition 
(LJDAM)  that  was  fielded  in  response  to  an  Urgent 
Operational  Need  (UON)  requirement.  DAMTC  expands  the 
UON  LJDAM’s  capability  from  a  simple  moving  target  to  a 
more  challenging  maneuvering  target.  A  maneuvering  target  is 
a  target  that  is  moving  but  changes  velocity,  direction,  or  both 
during  the  time  it  is  engaged. 

•  The  Navy  conducted  operational  testing  between 
November  2011  and  April  2012  resulting  in  a  Full-Rate 
Production  decision  in  June  2012. 

•  The  Navy  dropped  22  LJDAM  weapons  in  FY12  during  the 
DAMTC  LJDAM  operational  testing. 

System 

•  The  JDAM  is  a  low-cost,  autonomously  controlled,  adverse 
weather,  accurate  guidance  kit  tailored  for  Air  Force/Navy 
general  purpose  bombs  to  include: 

-  2,000-pound  Mk  84  and  BLU-109  bombs 

-  1,000-pound  Mk  83  and  BLU-110  bombs 

-  500-pound  Mk  82,  BLU-111,  BLU-126,  and  BLU-129/B 
bombs 

-  A  GPS-aided  inertial  navigation  system  that  provides 
primary  guidance  to  the  weapon.  Augmenting  the  JDAM 
inertial  navigation  system  with  GPS  signals  enhances 
accuracy. 

•  Guidance  and  control  designs  enable  accuracy  of  less  than 

5  meters  when  GPS  is  available  and  less  than  20  meters  when 
GPS  is  absent  or  jammed  after  release. 

•  The  LJDAM  provides  the  capability  to  attack  moving  targets 
by  enabling  such  targets  to  be  illuminated  with  laser  energy 
that  LJDAM’s  seeker  detects  and  tracks.  In  addition  to 
retaining  the  precision  of  JDAM  when  used  against  stationary 
targets,  the  LJDAM  provides  precise  laser  target  designation 
to  eliminate  Target  Location  Error,  ability  to  operate  beneath 
a  cloud  layer,  and  the  ability  to  select  weapon  impact  angle 

in  combination  with  laser-guided  precision.  LJDAM’s  laser 
guidance  allows  for  self-lasing  by  the  engaging  aircraft  or 
buddy-lasing  by  another  aircraft  or  a  ground-party. 

•  The  Navy  established  DAMTC  as  a  program  of  record  in 
February  2010  and  selected  LJDAM  as  the  non-developmental 
material  solution. 

Mission 

•  Combatant  Commanders  use  JDAMs  employed  by  fighter, 
attack,  and  bomber  aircraft,  to  engage  targets  day  or  night,  in 


all  weather  at  the  strategic,  operational,  and  tactical  levels  of 
warfare. 

•  Combatant  Commanders  employ  JDAM  against  fixed  and 
relocatable  soft  and  hard  targets,  to  include  command  and 
control  facilities,  airfields,  industrial  complexes,  logistical  and 
air  defense  systems,  lines  of  communication,  and  all  manner 
of  battlefield  forces  and  equipment. 

•  Combatant  Commanders  employ  the  UON  LJDAM  to  engage 
stationary  targets  using  JDAM-type  tactics,  as  well  as  to 
reactively  engage  stationary  and  moving  targets. 

•  The  Navy  and  Marine  Corps  intend  to  use  the  moving  and 
maneuvering  target  capability  of  the  DAMTC  LJDAM 
for  Close  Air  Support,  Strike  Coordination  and  Armed 
Reconnaissance,  and  Time  Sensitive  Target  missions  to  strike 
armored  and  unarmored  vehicles,  both  maneuvering  and 
stationary,  due  to  their  potential  to  start  maneuvering. 

Major  Contractor 

The  Boeing  Company,  Integrated  Defense  Systems  -  St.  Charles, 

Missouri 
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Activity 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  initiated  operational  testing  in  November  2011. 
Testing  incorporated  22  DAMTC  LJDAM  weapon  deliveries 
using  F/A-18  and  AV-8B  aircraft  from  VX-9  against  18 
moving  and  4  stationary  targets.  The  Navy  completed  IOT&E 
testing  in  April  2012. 

•  The  Navy  established  DAMTC  as  a  program  of  record  in 
February  2010,  selecting  LJDAM  as  the  non-developmental 
material  solution.  An  updated  laser  sensor  lens  necessitated 
limited  integrated  and  developmental  testing  prior  to 
commencement  of  operational  testing. 

•  The  LJDAM  employment  history  showed  significant 
degradation  of  the  lens  when  deployed  in  harsh  environments 
(such  as  Afghanistan).  The  Navy  initiated  a  search  for  a 
replacement  material  and  Boeing  developed  a  Sapphire  lens  to 
replace  the  existing  glass  lens. 

•  The  Navy  conducted  a  six-weapon  developmental  test  phase, 
using  side-by-side  comparison  testing  between  the  two  lens 
types,  immediately  prior  to  the  operational  test  phase  to 
ensure  the  Sapphire  lens  does  not  negatively  affect  system 
effectiveness. 

•  COTF  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

Assessment 

•  Preliminary  developmental  test  results  using  the  new  Sapphire 
lens  indicate  highly  comparable  sensor  detection  ranges  to  the 
previous  material. 

•  The  Sapphire  material  is  expected  to  provide  improved 
reliability  in  harsh  environments  during  its  intended  service 
life.  Laboratory  environmental  testing  and  flight  test  results 
demonstrated  Sapphire  performance  met  system-level 
requirements.  However,  due  to  limited  flight  testing,  data  are 
currently  insufficient  to  assess  the  reliability  of  the  new  lens 
material. 

•  Operational  test  results  showed  the  DAMTC  LJDAM  to  be 
operationally  effective  in  the  self-lasing  mode  against  targets 
that  both  moved  and  maneuvered  during  weapon  flight.  The 
DAMTC  LJDAM’s  demonstrated  accuracy  was  5.8  meters, 
meeting  the  6-meter  accuracy  requirement.  Accuracy 
using  the  buddy-lasing  mode  was  poor;  however,  the  range 
restrictions  and  target  limitations  that  prevailed  during  the 
four  buddy-lasing  weapon  deliveries  (and  not  present  during 


the  self-lasing  deliveries)  are  believed  to  have  contributed  to 
the  poor  results.  Demonstrated  accuracy  is  sufficient  to  assure 
lethal  effects  against  a  DAMTC  LJDAM  maneuvering  vehicle 
or  stationary  targets. 

•  Operational  test  results  showed  the  DAMTC  LJDAM  to 
be  operationally  suitable.  Using  both  operational  test  and 
integrated  test  phase  weapons  to  estimate  reliability  resulted 
in  a  material  reliability  of  98  percent  (46  of  47),  exceeding  the 
90  percent  requirement.  The  only  hardware  failure  was  a  live 
weapon  that  did  not  detonate  upon  impact.  An  overall  system 
reliability  of  77  percent  is  the  result  of  three  different  operator 
error  failures  and  a  single  large  miss  distance  of  unknown 
origin.  It  should  be  noted  that  there  is  not  a  requirement 
specified  for  system  reliability. 

•  Operational  testing  highlighted  two  deficiencies  related 
to  human  factors.  The  first  is  the  dense  wiring  inside  the 
tail-kit  of  the  LJDAM  weapons  that  makes  verifying  fuze 
arming  and  function  settings  extremely  difficult,  especially  at 
night.  The  umbilical  wire  bundles  result  in  a  very  crowded 
tail  compartment  making  it  difficult  to  read  the  settings. 
Workarounds  were  all  deemed  unacceptable  because  they 
either  prevent  the  aircrew  from  positively  confirming  proper 
fuze  settings  or  interfere  with  the  configuration  of  the  original 
assembly  of  a  live  weapon. 

•  The  second  human  factors  deficiency  is  the  high  cockpit 
workload  associated  with  delivering  a  DAMTC  LJDAM, 
though  this  is  comparable  with  the  high  workloads  found 
with  other  laser-guided  weapons.  Some  reduction  should 
be  achievable  through  aircraft  Operational  Flight  Program 
improvements  but  targeting  pod  limitations  appear  to  drive 
most  of  the  inherent  workload. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  completed 
the  FY 1 1  recommendation. 

•  FY  12  Recommendations.  The  Navy  should: 

1 .  Conduct  additional  testing  using  buddy-lasing  from  rear 
aspect  geometries  to  distinguish  between  the  effects  of  the 
range  restrictions  and  target  limitations  and  the  use  of  the 
buddy-lasing  tactic  on  the  DAMTC  LJDAM’s  accuracy. 

2.  Re-design  the  wiring  bundle  in  the  weapon’s  tail 
compartment  to  enable  an  accurate  visual  pre-flight  check 
of  the  weapon’s  fuze  settings. 
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E-2D  Advanced  Hawkeye 


Executive  Summary 

•  E-2D  IOT&E  occurred  from  February  to  September  2012. 

The  Navy  plans  to  seek  a  Full-Rate  Production  decision  from 
USD(AT&L)  in  January  2013. 

•  IOT&E  included  over  600  flight  hours  in  overland,  littoral, 
and  over  water  environments.  During  testing,  the  E-2D 
tracked  drone  and  traditional  fighter-size  aircraft  targets.  The 
test  squadron  embarked  aircraft  aboard  a  Navy  aircraft  carrier 
for  at-sea  testing,  including  carrier  suitability,  interoperability, 
and  supportability. 

•  The  Navy’s  Cooperative  Engagement  Capability  (CEC)  was 
supposed  to  complete  follow-on  testing  in  conjunction  with 
the  E-2D  IOT&E;  however,  CEC  developmental  delays  forced 
the  de-coupling  of  these  two  events.  As  such,  during  IOT&E 
the  Navy  did  not  test  the  operationally  representative  software 
configuration  of  the  CEC  system  in  the  E-2D. 

•  IOT&E  was  adequate  to  assess  the  E-2D  Advanced  Hawkeye’s 
Air-to-Air  and  Strike  Warfare  mission  performance. 

•  E-2D  demonstrated  improved  surveillance  capabilities  relative 
to  the  E-2C.  Test  aircrews  identified  performance  shortfalls 
with  operator  workload  in  dynamic,  complex,  high-target 
density  environments. 

•  DOT&E  was  unable  to  assess  during  IOT&E  the  final  E-2D 
CEC  system  configuration  performance  -  a  critical  enabler  for 
the  Theater  Air  and  Missile  Defense  (TAMD)  mission. 

•  The  E-2D  Advanced  Hawkeye  demonstrated  the  ability 
to  track  cruise  missile-sized  targets  during  developmental 
and  operational  testing.  While  the  system  provided 
improved  overland  performance,  the  Navy  needs  to  continue 
development  efforts  to  provide  a  robust  capability  in  all 
overland  environments. 

•  Based  on  IOT&E  reliability  and  availability  data,  DOT&E  has 
identified  shortfalls  on  some  radar  reliability  and  weapon  system 
availability  metrics,  as  well  as  immature  training  systems. 

System 

•  The  E-2D  Advanced  Hawkeye  is  a  carrier-based  Airborne 
Early  Warning  and  Command  and  Control  aircraft. 

•  Significant  changes  to  this  variant  of  the  E-2  include  upgraded 
engines  to  provide  increased  electrical  power  and  cooling 


relative  to  current  E-2C  aircraft;  a  strengthened  fuselage  to 
support  increased  aircraft  weight;  replacement  of  the  radar 
system,  the  communications  suite,  and  the  mission  computer; 
and  the  incorporation  of  an  all-glass  cockpit,  which  permits 
the  co-pilot  to  act  as  a  tactical  fourth  operator  in  support  of  the 
system  operators  in  the  rear  of  the  aircraft. 

•  The  radar  upgrade  replaces  the  E-2C  mechanically  scanned 
radar  with  a  phased-array  radar  that  has  combined  mechanical 
and  electronic  scan  capabilities. 

•  The  upgraded  radar  is  intended  to  provide  significant 
improvement  in  Hawkeye  littoral  and  overland  detection 
performance  and  TAMD  capabilities. 

•  The  total  E-2D  Advanced  Hawkeye  system  includes  all 
simulators,  interactive  computer  media,  and  documentation 
to  conduct  maintenance,  as  well  as  aircrew  shore -based  initial 
and  follow-on  training. 

Mission 

The  Combatant  Commander,  whether  operating  from  the  aircraft 

carrier  or  from  land,  will  use  the  E-2D  Advanced  Hawkeye  to 

accomplish  the  following  missions: 

•  Theater  air  and  missile  sensing  and  early  warning 

•  Battlefield  management,  command,  and  control 

•  Acquisition,  tracking,  and  targeting  of  surface  warfare  contacts 

•  Surveillance  of  littoral  area  objectives  and  targets 

•  Tracking  of  strike  warfare  assets 

Major  Contractor 

Northrop  Grumman  Aerospace  Systems  -  Bethpage,  New  York 


Activity 

•  The  Navy  conducted  the  E-2D  IOT&E  from 
February  to  September  2012  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan.  The  Navy  plans  to  seek  a  Full-Rate  Production  decision 
from  USD(AT&L)  in  2QFY13. 

•  IOT&E  testing  included  over  600  flight  hours  in  overland, 
littoral,  and  over  water  environments.  During  testing,  the 


E-2D  tracked  drone  and  traditional  fighter-size  aircraft  targets. 
The  test  squadron  conducted  over  200  tracking  events  to 
assess  Air-to-Air  and  Strike  Warfare  mission  performance. 

•  The  test  squadron  embarked  aircraft  aboard  a  Navy  aircraft 
carrier  for  at-sea  testing,  including  carrier  suitability, 
interoperability,  and  supportability.  This  testing  included 
operations  from  the  aircraft  carrier  in  support  of  a  Joint  Task 
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Force-level  exercise  as  well  as  additional  interoperability 
testing  during  Carrier  Strike  Group  certification  training, 
which  involved  all  current  carrier  air  wing  capable  aircraft. 

The  test  squadron  conducted  E-2D  operations  while 
participating  in  joint  Large  Force  Exercises  at  Nellis  AFB, 
Nevada,  and  Fallon  Naval  Air  Station,  Nevada.  Additionally, 
the  test  squadron  conducted  over-water  TAMD  tracking 
exercises  operating  with  Navy  ships  off  the  California  coast. 

•  The  Navy’s  CEC  program  planned  to  complete  follow-on 
testing  in  conjunction  with  the  E-2D  IOT&E;  however,  CEC 
developmental  delays  forced  the  de-coupling  of  these  two 
events.  As  such,  during  IOT&E  the  Navy  did  not  test  the 
operationally  representative  software  configuration  of  the 
CEC  system  in  the  E-2D.  The  Navy  addressed  the  flaws  in 
the  CEC  system  during  the  summer  of  2012  with  new  system 
software  loads  for  both  the  E-2D  mission  system  and  the 
CEC  system  and  entered  follow-on  test  for  the  CEC  system  in 
October  2012. 

Assessment 

•  IOT&E  was  adequate  to  assess  the  E-2D  Advanced 
Hawkeye’s  legacy  mission  performance.  DOT&E  expects  to 
publish  a  report  assessing  the  E-2D  IOT&E  performance  in 
January  2013. 

•  E-2D  was  found  to  have  improved  surveillance 
capabilities  relative  to  the  E-2C.  Test  aircrews  identified 
performance  shortfalls  with  operator  workload  in  dynamic, 
high-target  density  environments  where  the  E-2D  mission 
system  erroneously  swapped  identification  labels  for 
crossing/ closely-spaced  aircraft  tracks.  Subsequently,  the 
tracks  required  manual  aircrew  re-labeling  in  the  mission 
system,  which  can  lead  to  operator  overload  and  loss  of 
situational  awareness. 

•  DOT&E  was  unable  to  assess  the  final  E-2D  CEC  system 
configuration  performance  -  a  critical  enabler  for  the  TAMD 
mission.  The  TAMD  mission,  which  is  the  primary  reason 
the  E-2D  is  being  procured,  will  not  be  fully  accessible  until 
the  CEC  system  is  successfully  tested  and  end-to-end  TAMD 
testing  is  completed  in  the  2015  timeframe,  shortly  before  the 
E-2D  Initial  Operational  Capability  declaration. 

•  The  E-2D  Advanced  Hawkeye  demonstrated  the  ability  to 
track  cruise  missile-sized  targets  during  developmental  and 
operational  testing.  While  the  system  provided  improved 
overland  performance,  the  Navy  needs  to  continue  radar 
and  mission  system  development  efforts  to  provide  a  robust 
capability  in  all  overland  environments. 

•  With  the  exception  of  CEC  testing  still  in  progress,  E-2D  voice 
and  datalink  communications  match  those  of  E-2C.  The  E-2D 
has  the  same  Passive  Detection  System  found  in  the  E-2C. 
DOT&E  assesses  this  capability  as  matching  that  of  the  current 
E-2C. 

•  The  E-2D  mission  planning  system,  the  Joint  Mission  Planning 
System  (JMPS),  is  not  currently  effective.  Operators  use 

the  JMPS  to  provide  a  means  of  importing  mission-planning 
data  into  the  E-2D  mission  computer  for  use  during  flight. 


There  are  JMPS  shortcomings  in  reliability  and  usability 
causing  the  mission-planning  process  to  be  cumbersome  and 
time-consuming.  Additionally,  there  is  a  deficiency  with  JMPS 
loading  the  proper  network  configuration  data  for  some  Link  16 
networks. 

•  Based  on  IOT&E  reliability  and  availability  data,  DOT&E 
has  identified  shortfalls  on  some  radar  reliability  and  weapon 
system  availability  metrics.  E-2D  maintainability  had  no 
significant  shortfalls  in  on-aircraft  maintenance  procedures, 
documentation,  or  support  equipment.  However,  the 
maintenance  training  system  for  the  E-2D  will  not  be  available 
until  July  2013  due  to  prior  acquisition  decisions.  The  lack  of  a 
maintenance  training  system  precludes  a  full  assessment  of  the 
maintainability  of  the  E-2D  system. 

•  DOT&E  could  not  assess  aircrew  training  due  to  the 
non-availability  of  the  E-2D  integrated  simulator,  which  will  be 
used  for  shore-based  initial  and  follow-on  training,  as  well  as 
aircrew  tactical  and  carrier  operating  procedures  proficiency. 

•  The  E-2D  aircraft  performed  nominally  during  at-sea 
operations.  The  current  E-2C  system  operates  in  a  four 
aircraft-per-squadron  configuration  as  opposed  to  the  E-2D, 
which  the  Navy  plans  to  operate  in  a  five  aircraft-per-squadron 
configuration.  The  E-2D  at-sea  testing  did  not  demonstrate 
fully  the  ability  to  logistically  support  the  proposed  five-aircraft 
E-2D  squadron  in  the  aircraft  carrier  environment.  The  limited 
number  of  at-sea  sorties  and  the  current  limited  spare  parts 
support  for  E-2D  precluded  a  full  at-sea  logistics  supportability 
assessment  of  the  five-aircraft  E-2D  squadron  concept. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  continues 
efforts  to  address  CEC  and  radar  performance  improvements  as 
recommended  in  FY 1 1 .  These  efforts  are  critical  to  achieving 
the  2015  E-2D  Initial  Operational  Capability  and  full  TAMD 
capability. 

•  FY12  Recommendations. 

1.  The  Navy  should  continue  to  improve  E-2D  CEC 
performance  following  completion  of  the  E-2D  CEC 
follow-on  test  in  January  2013. 

2.  The  E-2D  program  should  investigate  potential  radar  and 
mission  system  performance  upgrades  to  improve  system 
performance  in  the  challenging  overland  arena. 

3.  The  E-2D  program  should  resolve  the  mission  system  track 
re-labeling  deficiency. 

4.  The  E-2D  program  should  work  with  the  JMPS  Program 
Office  to  provide  for  E-2D  aircrews  an  effective  mission 
planning  system. 

5.  The  E-2D  program  should  aggressively  focus  on 
implementing  the  full  logistics  posture  to  support  the 
E-2D  five-aircraft  squadron  concept  in  the  aircraft  carrier 
environment. 

6.  The  E-2D  Program  Office  should  continue  to  improve  radar 
and  overall  weapon  system  reliability  and  availability. 
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Enhanced  Combat  Helmet  (ECH) 


Executive  Summary 

•  The  Enhanced  Combat  Helmet  (ECH)  successfully  met 
its  ballistic  and  non-ballistic  requirements  during  its  First 
Article  Test  (FAT).  However,  while  the  ECH  protects  against 
perforation  by  the  specified  small  arms  threat,  it  does  not 
provide  a  significant  overall  improvement  in  operational 
capability  over  currently  fielded  helmets  against  that  threat. 
The  ECH  provides  improved  fragmentation  protection 
compared  to  the  fielded  Advanced  Combat  Helmet  and  the 
Light  Weight  Helmet  (LWH). 

•  Subsequent  to  FAT,  during  Engineering  Change  Proposal 
(ECP)  testing  of  helmets  manufactured  using  new  machinery 
and  tooling  intended  to  provide  increased  helmet  production 
capacity,  the  ECH  experienced  unexpected  ballistic  failures. 

•  Ceradyne  (the  manufacturer)  indicated  it  had  identified  the 
reasons  for  these  ballistic  failures  and  implemented  corrective 
actions.  The  ECH  Program  Office  conducted  additional 
testing  to  verify  the  effectiveness  of  Ceradyne ’s  corrective 
actions.  The  ECH  continued  to  experience  unexpected 
ballistic  failures  during  this  testing. 

•  The  ECH  Program  Office  and  Ceradyne  continue  to 
investigate  the  reasons  for  the  ballistic  failures. 

•  The  ECH  Program  Office  has  delayed  production  and  fielding 
of  the  ECH  pending  identification  and  correction  of  the 
reasons  for  the  ballistic  failures.  The  ECH  Program  Office 
projects  testing  to  determine  the  effectiveness  of  corrective 
actions  will  begin  in  early  2013. 

System 

•  The  Marine  Corps  developed  the  ECH  in  response  to  a  2009 
Urgent  Statement  of  Need  to  produce  a  helmet  that  provides 
ballistic  protection  from  energetic  fragments  and  selected 
small  anus  ammunition,  yet  maintains  all  other  characteristics 
of  the  Marine  Corps’  LWH  and  the  Army’s  Advanced  Combat 
Helmet  (ACH). 

•  The  ECH  is  compatible  with  and  is  typically  worn  in 
conjunction  with  other  components  of  infantry  combat 
equipment  such  as  body  armor  systems,  protective  goggles, 
night  vision  equipment,  and  a  camouflage  fabric  helmet  cover. 
This  new  helmet  is  intended  to  provide  Marines  and  Soldiers 
improved  protection  compared  to  the  currently  fielded  LWH 
and  ACH. 


•  The  ECH  consists  of  a  ballistic  protective  shell,  a  pad 
suspension  system,  and  a  4-point  chin  strap/nape  strap 
retention  system.  Unlike  the  ACH  and  LWH  helmets,  which 
are  constructed  with  aramid  fibers,  the  ECH  is  constructed 
using  ultra-high-molecular-weight  polyethylene  fibers. 

Mission 

Forces  equipped  with  the  ECH  will  rely  on  the  helmet  to  provide 
ballistic  protection  from  selected  small  arms  ammunition  and 
fragmentation  when  engaged  with  enemy  combatants  during 
tactical  operations  in  accordance  with  applicable  tactics, 
techniques,  and  procedures. 

Major  Contractor 

Ceradyne,  Inc.  -  Costa  Mesa,  California 


Activity 

•  The  Marine  Corps  conducted  FAT  II  during  November 

and  December  2011  in  accordance  with  the  DOT&E-approved 
test  plan.  Following  successful  completion  of  this  FAT,  the 
Marine  Corps  conducted  Full-Up  System-Level  live  fire 


tests  against  both  direct  and  indirect  fire  threats  between 
December  2011  and  February  2012. 

•  In  April  2012,  the  ECH  Program  Office  began  testing  ECPs 
to  assess  if  helmets  produced  using  new  machinery,  tooling, 
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or  hardware  (necessary  to  expand  ECH  production  capability) 
continued  to  meet  performance  requirements  prior  to 
incorporating  these  new  items  into  the  production  process. 

•  Due  to  unexpected  ballistic  failures  during  ECP  testing, 
Ceradyne  began  an  analysis  of  the  causes  for  the  failures,  as 
well  as  actions  required  to  correct  the  reasons  for  the  failures. 
Ceradyne  implemented  corrective  actions  it  thought  would 
be  effective  and  submitted  newly-manufactured  helmets 

for  testing  that  began  in  late  June  2012.  The  ECH  Program 
Office  designed  the  testing  to  verify  that  the  corrective 
actions  addressed  the  reasons  for  the  ballistic  failures,  and 
that  the  helmets  met  ballistic  requirements.  However,  the 
helmet  continued  to  experience  ballistic  failures  during  this 
verification  testing. 

•  In  July  2012,  the  ECH  Program  Office  assembled  a  team  to 
assist  Ceradyne  with  the  subsequent  failure  analysis.  Both 
the  ECH  Program  Office  and  Ceradyne  continue  to  conduct 
root  cause  analysis  to  determine  the  reasons  for  the  ballistic 
failures. 

•  The  ECH  Program  Office  currently  projects  testing  will  begin 
in  early  2013  to  verify  Ceradyne  has  corrected  the  reasons 
for  the  ballistic  failures  and  that  the  helmets  continue  to  meet 
requirements. 

Assessment 

•  Although  the  ECH  protects  against  perforation  by  the 
specified  small  arms  threat,  it  does  not  provide  a  significant 
overall  improvement  in  operational  capability  over  currently 
fielded  helmets  against  that  threat.  It  is  unlikely  to  provide 
meaningful  protection  against  this  small  arms  threat  over  a 
significant  portion  of  the  threat’s  effective  range.  However, 
the  ECH  does  provide  improved  penetration  protection  against 
fragments  relative  to  currently  fielded  helmets.  The  ECH  met 
all  ballistic  performance  requirements. 

•  In  stopping  high-energy  threats,  the  helmet  absorbs  the 
projectile  energy  by  deforming  inward  toward  the  skull.  It  is 
unknown,  definitively,  whether  the  ECH  provides  protection 
against  injury  when  the  deforming  helmet  impacts  the  head. 
There  is,  however,  reason  to  be  concerned  because  the 
deformation  induced  by  the  impact  of  a  non-perforating  small 
arms  threat  exceeds  accepted  deformation  standards  across 
most  of  the  threat’s  effective  range. 

•  There  are  no  definitive  medical  criteria  or  analytic  methods  to 
correlate  the  extent  of  helmet  deformation  to  injury.  However, 
according  to  the  Armed  Forces  Medical  Examiner’s  Office, 
even  absent  medical  studies  definitively  correlating  helmet 


deformation  with  specific  injury,  the  deformation  observed 
during  testing  represents  significant  blunt  force  and/or 
penetrating  trauma  to  the  head  that  could  be  lethal. 

•  While  the  ECH  met  the  stated  resistance  to  the  perforation 
requirement  against  the  specified  small  arms  threat,  one  helmet 
shot  location  was  more  prone  to  small  arms  perforations  than 
others.  The  ECH  Program  Office  plans  to  implement  an  ECP 
to  address  this  problem. 

•  Structural  degradation  as  a  result  of  prolonged  temperature  and 
humidity  exposure  may  be  a  concern  for  the  ECH.  Published 
data  document  the  degradation  of  ballistic  performance  in 
ultra-high-molecular-weight  polyethylene  materials,  but  the 
long-term  performance  of  the  ECH’s  specific  ballistic  material 
is  unknown. 

•  The  ECH  Program  Office  has  contracted  to  procure  ECHs  in 
production  lots  of  mixed  helmet  sizes.  There  is  no  assurance 
that  testing  of  these  mixed  lots  will  include  sufficient  numbers 
of  the  individual  sizes  produced  within  a  lot  to  draw  valid 
conclusions  from  the  test  results. 

Recommendations 

•  Status  of  Previous  Recommendations.  There  were  no  previous 
recommendations  for  this  program. 

•  FY12  Recommendations.  The  ECH  Program  Office  should: 

1.  Conduct  adequate  testing  to  ensure  the  failures  observed 
during  ECP  testing  are  identified  and  corrected.  In  addition, 
they  should  test  and  implement  the  planned  ECP  to  address 
small  arms  perforation  concerns  at  the  one  shot  location. 
Until  the  ECP  addressing  the  small  arms  perforation 
concern  is  implemented,  the  ECH  Program  Office  should 
conduct  adequate  lot  acceptance  testing  to  ensure  the  helmet 
provides  adequate  protection  from  small  arms  perforation  at 
all  impact  locations. 

2.  Carefully  monitor  the  results  of  lot  acceptance  testing  when 
ECH  production  begins  for  indications  of  variations  in  the 
manufacturing  process  that  could  affect  the  ECH’s  ballistic 
protection. 

3.  Improve  ECH  protection  by  reducing  the  amount  of  helmet 
deformation  caused  by  non-perforating  small  arms  impacts, 
as  improvements  in  materials  and  manufacturing  processes 
permit. 

4.  Conduct  testing  to  determine  whether  long-term  exposure  to 
elevated  temperatures  and  humidity  degrades  ECH  ballistic 
performance. 

5.  Procure  ECH  in  production  lots  of  a  single  helmet  size  to 
ensure  adequate  lot  acceptance  testing. 
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F/A-18E/F  Super  Hornet  and  EA-18G  Growler 


Executive  Summary 

•  In  FY11,  the  Navy  conducted  Software  Qualification 
Testing  (SQT)  of  System  Configuration  Set  (SCS)  H6E  for 
F/A-18E/F  Super  Flornet  and  EA-18G  Growler  aircraft, 

SQT  of  SCS  23X  for  early-model  F/A-18E/F  aircraft,  and  a 
second  FOT&E  period  for  the  APG-79  Active  Electronically 
Scanned  Array  (AESA)  radar.  DOT&E  issued  individual 
classified  reports  on  this  testing  for  both  F/A-18E/F  and 
EA-18G  in  FY12. 

•  The  APG-79  AESA  radar  provides  improved  performance 
relative  to  the  legacy  APG-73  radar;  however,  operational 
testing  did  not  demonstrate  a  statistically  significant 
difference  in  mission  accomplishment  between  F/A-18E/F 
aircraft  equipped  with  AESA  and  those  equipped  with  the 
legacy  radar. 

•  While  SCSs  H6E  and  23X  demonstrate  acceptable 
suitability,  the  AESA  radar’s  reliability  continues  to 
suffer  from  software  instability.  The  radar’s  failure  to 
meet  reliability  requirements  and  poor  built-in  test  (BIT) 
performance  remain  as  shortfalls  from  previous  test  and 
evaluation  periods. 

•  Overall,  the  F/A-18E/F  Super  Hornet  weapon  system 
is  operationally  effective  and  suitable  for  most  threat 
environments.  However,  the  platform  is  not  operationally 
effective  for  use  in  certain  threat  environments,  the  specifics 
of  which  are  addressed  in  the  DOT&E  FY12  classified 
report. 

•  The  EA-18G  Growler  weapon  system  is  operationally 
effective  and  operationally  suitable. 

•  The  Navy  is  conducting  F/A-18E/F  and  EA-18G  SCS 
H8E  SQT  in  two  phases.  Phase  I  operational  testing 
commenced  in  4QFY12  and  is  scheduled  to  complete  in 
1QFY13.  The  Navy  expects  to  conduct  Phase  II  testing 
during  2Q  -  3QFY13.  DOT&E  will  issue  a  single  report 
covering  both  H8E  phases  after  the  completion  of  Phase  II  in 
4QFY13. 

System 

F/A-18E/F  Super  Hornet 

•  The  Super  Hornet  is  the  Navy’s  premier  strike -fighter 
aircraft  that  replaces  earlier  F/A-18  variants  in  carrier  air 
wings.  The  F/A-18E  is  a  single-seat  aircraft  while  the 

F  model  has  two  seats. 

•  F/A-18E/F  Lot  26+  aircraft  provide  functionality  essential 
for  integrating  all  Super  Hornet  Block  2  hardware 
upgrades,  which  include: 

-  Single  pass  multiple  targeting  for  GPS-guided  weapons 

-  Use  of  off-board  target  designation 

-  Improved  datalink  target  coordination  precision 

-  Implementation  of  air-to-ground  target  points 


•  Additional  systems  include: 

-  APG-73  or  APG-79  radar 

-  Advanced  Targeting  and  Designation  Forward-Looking 
Infrared  System 

-  AIM-9  infrared-guided  missiles  and  AIM-120  and 
AIM-7  radar-guided  missiles 

-  Shared  Reconnaissance  Pod 

-  Multi-functional  Information  Distribution  System  for 
Link  16  tactical  datalink  connectivity 

-  Joint  Helmet  Mounted  Cueing  System 

-  Integrated  Defensive  Electronic  Countermeasures 

EA-18G  Growler 

•  The  Growler  is  the  Navy’s  land-  and  carrier-based,  radar 
and  communication  jamming  aircraft. 

•  The  two-seat  EA-18G  replaces  the  four-seat  EA-6B.  The 
new  ALQ-218  receiver,  improved  connectivity,  and  linked 
displays  are  the  primary  design  features  implemented  to 
reduce  the  operator  workload  in  support  of  the  EA-18G’s 
two-person  crew. 

•  The  Airborne  Electronic  Attack  (AEA)  system  includes: 

-  Modified  E A-6B  Improved  Capability  III  ALQ-2 1 8 
receiver  system 

-  Advanced  crew  station 

-  Legacy  ALQ-99  j  amming  pods 

-  Communication  Countermeasures  Set  System 

-  Expanded  digital  Link  1 6  communications  network 

-  Electronic  Attack  Unit 

-  Interference  Cancellation  System  that  supports 
communications  while  jamming 
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-  Satellite  receive  capability  via  the  Multi-mission 
Advanced  Tactical  Terminal 

•  Additional  systems  include: 

-  APG-79  AESA  radar 

-  Joint  Helmet  Mounted  Cueing  System 

-  High-speed  Anti-Radiation  Missile 

-  AIM- 120  radar-guided  missiles 
System  Configuration  Set  (SCS)  Software 

•  Growler  and  Super  Hornet  aircraft  employ  SCS  operational 
software  to  enable  major  combat  capabilities.  All  EA-18Gs 
and  Block  2  F/A-18s  (production  Lot  26  and  beyond) 

use  high-order  language  or  “H-series”  software,  while 
F/A-18E/F  prior  to  Lot  26  and  all  legacy  F/A-18  A/B/C/D 
aircraft  use  “X-series”  software.  The  current  fleet-release 
software  are  H6E  and  23X,  respectively. 

Mission 

•  Combatant  Commanders  use  the  F/A-18E/F  to: 

-  Conduct  offensive  and  defensive  air  combat  missions 

-  Attack  ground  targets  with  most  of  the  U.S.  inventory  of 
precision  and  non-precision  weapon  stores 


-  Provide  in-flight  refueling  for  other  tactical  aircraft 

-  Provide  the  fleet  with  an  organic  tactical  reconnaissance 
capability 

•  Combatant  Commanders  use  the  EA-18G  to: 

-  Support  friendly  air,  ground,  and  sea  operations  by 
countering  enemy  radar  and  communications 

-  Jam  integrated  air  defense  systems 

-  Support  non-integrated  air  defense  missions  and  emerging 
non-lethal  target  sets 

-  Enhance  crew  situational  awareness  and  mission 
management 

-  Enhance  connectivity  to  national,  theater,  and  tactical  strike 

assets 

-  Provide  enhanced  lethal  suppression  through  accurate 
High-speed  Anti-Radiation  Missile  targeting 

-  Provide  the  EA-18G  crew  air-to-air  self-protection  with  the 
AIM- 120 

Major  Contractor 

The  Boeing  Company,  Integrated  Defense  Systems  -  St.  Louis, 
Missouri 


Activity 

•  DOT&E  reported  on  APG-79  radar  IOT&E  in  FY07,  assessing 
it  as  not  operationally  effective  or  suitable  due  to  significant 
deficiencies  in  tactical  performance,  reliability,  and  BIT 
functionality. 

•  The  Navy  conducted  APG-79  radar  FOT&E  in  FY09  in 
conjunction  with  SCS  H4E  SQT.  The  Navy’s  Commander, 
Operational  Test  and  Evaluation  Force  subsequently  reported 
that  significant  deficiencies  remained  for  both  APG-79  AESA 
performance  and  suitability;  DOT&E  concurred  with  this 
assessment. 

•  Concurrent  with  SQT  for  SCSs  H6E  and  23X,  the  Navy 
conducted  a  second  APG-79  radar  FOT&E  period  in  FY11. 
The  Navy  conducted  the  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  (TEMP) 
and  test  plan.  DOT&E  issued  a  classified  report  on  this  testing 
in  FY12;  finding  that  the  Super  Hornet  made  incremental 
improvements,  but  still  retained  important  deficiencies. 

•  The  Navy  is  conducting  F/A-18E/F  and  EA-18G  SCS 
H8E  SQT  in  two  phases.  Phase  I  operational  testing  was 
originally  scheduled  for  1QFY12,  but  due  to  multiple  software 
anomalies  during  developmental  testing,  the  Navy  postponed 
the  Operational  Test  Readiness  Review  until  3QFY12.  H8E 
Phase  I  operational  testing  commenced  in  4QFY12  and  is 
scheduled  to  complete  in  1QFY13.  The  Navy  is  conducting 
the  testing  in  accordance  with  the  DOT&E-approved  TEMP 
and  test  plan. 

•  The  Navy  expects  to  conduct  H8E  Phase  II  operational  testing 
during  2Q-3QFY13. 

•  SCS  H8E  testing  does  not  include  an  end-to-end 
multi- AIM- 120  missile  shot.  This  capability  is  a  Navy 


operational  requirement  not  previously  demonstrated  or 
successfully  tested.  The  Navy  tentatively  plans  to  conduct  a 
multi-missile  shot  in  conjunction  with  SCS  H12E  testing. 

Assessment 

•  The  APG-79  AESA  radar  demonstrated  marginal 
improvements  since  the  previous  FOT&E  period  and  provides 
improved  performance  relative  to  the  legacy  APG-73 

radar.  However,  operational  testing  does  not  demonstrate  a 
statistically  significant  difference  in  mission  accomplishment 
between  F/A-18E/F  aircraft  equipped  with  AESA  and  those 
equipped  with  the  legacy  radar. 

•  Full  development  of  AESA  electronic  warfare  capability 
remains  deferred  to  later  software  builds. 

•  While  SCSs  H6E  and  23X  demonstrate  acceptable  suitability, 
the  AESA  radar’s  reliability  continues  to  suffer  from  software 
instability  despite  software  upgrades.  The  radar’s  failure 

to  meet  reliability  requirements  and  poor  BIT  performance 
remain  as  shortfalls  from  previous  test  and  evaluation  periods. 

•  Overall,  the  F/A-18E/F  Super  Hornet  weapon  system 
is  operationally  effective  and  suitable  for  most  threat 
environments.  However,  the  platform  is  not  operationally 
effective  for  use  in  certain  threat  environments,  the  details  of 
which  are  addressed  in  DOT&E’s  classified  report. 

•  The  EA-18G  Growler  weapons  system  is  operationally 
effective  and  operationally  suitable. 

•  DOT&E  will  report  on  Super  Hornet  and  Growler  SCS  H8E 
capability  improvements  after  both  Phase  I  and  II  operational 
testing  are  complete  in  4QFY13. 
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Recommendations 

•  Status  of  Previous  Recommendations. 

-  The  Navy  made  minimal  progress  addressing  F  Y 1 1 
F/A-18E/F  recommendations.  Recommendations  to 
continue  to  improve  APG-79  AESA  reliability  and  BIT 
functionality,  to  conduct  an  operationally  representative 
end-to-end  missile  shot  to  demonstrate  APG-79  radar  and 
current  SCS  ability  to  support  multi-AIM-120  engagement, 
and  to  develop  and  characterize  the  APG-79  AESA’s  full 
electronic  warfare  capability  remain  valid. 


-  The  Navy  satisfactorily  addressed  three  of  seven 
FY11  EA-18G  recommendations.  Recommendations 
to  improve  aircraft  maintainability  and  BIT  software 
maturity,  to  improve  ALQ-2 18  and  ALQ-99  maintenance 
documentation  and  diagnostic  tools,  and  to  assess  the 
need  for  a  more  capable  threat  range  at  Whidbey  Island, 
Washington,  remain  valid. 

•  FY12  Recommendations.  None. 
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Integrated  Defensive  Electronic  Countermeasures 

(IDECM) 


Executive  Summary 

•  DOT&E  approved  the  classified  Integrated  Defensive 
Electronic  Countermeasures  (IDECM)  Block  IV  (IB-4)  Test 
and  Evaluation  Master  Plan  (TEMP)  in  January  2012. 

•  The  Navy  authorized  the  first  lot  buy  of  IB-4  systems  in 
March  2012  following  its  In-Process  Review  (IPR)  #3. 

At  that  time,  DOT&E  assessed  that  the  system  was  nine 
months  behind  schedule  and  less  mature  than  planned.  No 
effectiveness  or  suitability  results  were  available  to  support  the 
lot  production  decision. 

•  Since  IPR  #3  in  March  2012,  the  program  has  been  delayed 
an  additional  three  months  and  the  operational  assessment 
(OA)  has  been  reduced  in  scope  because  of  IB-4  software 
immaturity  and  unavailability  of  laboratory  resources. 

•  The  IB-4  OA  began  September  2012  and  is  expected  to 
conclude  by  December  2012.  It  includes  flight  tests  and  a 
laboratory  test.  The  Navy  has  made  progress  in  hardware 
testing  and  software  development,  and  is  resolving 
deficiencies  at  an  increasing  rate  since  the  IPR  #3.  However, 
the  Navy  continues  to  discover  system  deficiencies  at  a  steady 
rate,  and  DOT&E  anticipates  that  the  program  will  need 
additional  time  to  resolve  system  deficiencies,  thus  extending 
the  test  schedule. 

System 

•  The  IDECM  system  is  a  radio  frequency,  self-protection 
electronic  countermeasure  suite  on  F/A-18  aircraft.  The 
system  is  comprised  of  onboard  and  off-board  components. 
The  onboard  components  receive  and  process  radar 
signals  and  can  employ  onboard  and/or  off-board  jamming 
components  in  response  to  identified  threats. 

•  There  are  four  IDECM  variants:  Block  I  (IB-1),  Block  II 
(IB-2),  Block  III  (IB-3),  and  Block  IV  (IB-4).  All  four 
variants  include  an  onboard  radio  frequency  receiver  and 
jammer. 

-  IB-1  (fielded  FY02)  combined  the  legacy  onboard 
receiver/jammer  (ALQ-165)  with  the  legacy  (ALE-50) 
off-board  towed  decoy. 


-  IB-2  (fielded  FY04)  combined  the  improved  onboard 
receiver/jammer  (ALQ-214)  with  the  legacy  (ALE-50) 
off-board  towed  decoy. 

-  IB-3  (fielded  FY11)  combines  the  improved  onboard 
receiver/jammer  (ALQ-214)  with  the  new  (ALE-55) 
off-board  fiber  optic  towed  decoy  that  is  more  integrated 
with  the  ALQ-214. 

-  IB-4  (currently  in  development)  is  intended  to  replace 
the  onboard  receiver/jammer  (ALQ-214(V)3)  with  a 
lightweight,  repackaged  onboard  jammer  (ALQ-214(V)4 
and  ALQ-214(V)5). 

•  The  F/A-18E/F  installation  includes  off-board  towed 
decoys.  The  F-18C/D  installation  includes  only  the  onboard 
receiver/jammer  components  and  not  the  towed  decoy. 

Mission 

•  Combatant  Commanders  will  use  IDECM  to  improve  the 
survivability  of  Navy  F/A-18  strike  aircraft  against  radio 
frequency-guided  threats  while  on  air-to-air  and  air-to-ground 
missions. 

•  The  Navy  intends  to  use  IB-3’s  and  IB-4’s  complex  jamming 
capabilities  to  increase  survivability  against  modern  radar 
guided  threats. 

Major  Contractors 

•  ALE-55:  BAE  Systems  -  Nashua,  New  Hampshire 

•  ALQ-2 14:  ITT  Electronic  Systems  -  Clifton,  New  Jersey 

•  ALE-50:  Raytheon  Electronic  Warfare  Systems  -  Goleta, 
California 


Activity 

IDECM  Block  III 


•  DOT&E  completed  its  IDECM  Block  III  IOT&E  report 
in  June  2011,  assessing  the  system  as  operationally 
effective  and  operationally  suitable  for  combat.  The 
Navy  authorized  IDECM  Block  III  full-rate  production  in 
July  2011. 


IDECM  Block  IV 

•  Developmental  laboratory  testing  began  in  September 
2011  at  the  Navy’s  Advanced  Weapons  Laboratory  system 
integration  lab  at  China  Lake,  California,  and  the  Navy’s 
Electronic  Combat  System  Evaluation  Laboratory  (ECSEL) 
at  Point  Mugu,  California. 


IDECM 
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•  DOT&E  approved  the  classified  IB-4  TEMP  in 
January  2012. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  completed  a  Developmental  Test  Assist 
assessment  in  February  2012,  which  consisted  primarily 
of  observations  of  contractor  and  Navy  laboratory 
testing.  COTF  noted  significant  hardware,  software,  and 
compatibility  shortfalls  based  on  its  observations  (details 
are  classified). 

•  The  Navy  approved  the  first  lot  production  of  IB-4  systems 
in  March  2012. 

•  The  Navy  conducted  developmental  ground-mount  open-air 
range  testing  at  the  Navy’s  Slate  Range  at  China  Lake, 
California,  and  at  the  Nevada  Test  and  Training  Range.  The 
Navy  began  developmental  risk  reduction  flight  testing  in 
May  2012  at  China  Lake,  California. 

•  The  Navy  began  IB-4  OA  testing  in  September  2012, 
consisting  of  flight  testing  at  the  Electronic  Combat  Range 
in  China  Lake  and  laboratory  testing  at  the  ECSEL  in 
Point  Mugu.  The  OA  was  reduced  in  scope  from  what  was 
originally  described  in  the  January  2012  TEMP.  The  Navy 
requested  a  delay  in  one  of  the  planned  laboratory  tests  to 
further  mature  IB-4  software,  and  a  national  priority  Air 
Force  program  took  precedence  over  IB-4  at  one  of  the 
laboratory  test  facilities.  DOT&E  approved  this  reduction 
in  scope.  The  Navy  will  complete  the  two  deferred  tests 
as  described  in  the  DOT&E-approved  test  plan  prior  to  the 
start  of  FOT&E  and  DOT&E  will  issue  an  FOT&E  report. 

•  The  Navy  plans  to  complete  a  redesign  of  the  1DECM 
receiver  near  the  end  of  the  FOT&E.  The  Program 
Office  expects  the  redesigned  receiver’s  effect  on  system 
performance  to  be  minimal;  DOT&E  will  work  with  the 
Program  Office  to  determine  the  scope  of  any  necessary 
additional  testing. 

•  With  each  future  IPR,  beginning  with  IPR  #3  (March  2012), 
the  Navy  is  authorizing  purchases  of  the  ALQ-214(V)4  for 
the  F/A-18  E/F  and  the  ALQ-214(V)5  for  the  F/A-18  C/D. 

Assessment 

•  At  the  IPR  #3,  IB-4  was  9  months  behind  the  schedule 
the  Navy  presented  at  the  hardware  critical  design  review 
30  months  earlier.  The  Navy  had  not  completed  several 
key  hardware  tests  and  had  not  started  system  effectiveness 
and  suitability  testing,  which  led  to  a  less  informed  IPR  #3. 
Deficiency  report  submissions  from  developmental  testing 
were  continuing  at  a  steady  rate,  with  the  number  of 
unresolved  deficiencies  outpacing  the  number  of  resolved 
deficiencies,  showing  lack  of  system  maturity. 

•  The  Navy  has  made  progress  in  hardware  testing  and  software 
development,  and  is  resolving  deficiencies  at  an  increasing 


rate  since  IPR  #3.  However,  the  Navy  continues  to  discover 
system  deficiencies  at  a  steady  rate,  and  DOT&E  anticipates 
that  the  program  will  need  additional  time  to  resolve  system 
deficiencies,  thus  extending  the  test  schedule.  This  will  require 
the  Navy  to  either  postpone  conducting  the  next  IPR  and  lot 
production  decision  (currently  scheduled  for  March  2013) 
or  again  make  a  lot  production  decision  with  much  less 
information  than  originally  intended. 

•  DOT&E  assessed  system  maturity  at  the  start  of  the  OA  test  as 
less  than  the  program  originally  planned.  No  suitability  data 
and  limited  effectiveness  data  were  available  to  support  the 
OA  test  readiness  review,  and  therefore  the  program  assumed 
increased  risk  of  inadequate  system  performance  during  the 
OA. 

•  The  Navy  has  significantly  reduced  the  time  period  between 
the  completion  of  all  testing  planned  for  the  OA  and  FOT&E, 
thus  leaving  little  time  to  correct  deficiencies  found  as  a  result 
of  testing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
adequately  addressed  three  of  the  nine  IDECM-specific 
recommendations  from  F  Y 1 1 .  The  Navy  has  partially 
addressed  an  additional  three  of  the  nine  recommendations 
and  further  activity  to  resolve  them  is  ongoing.  The  three 
recommendations  that  may  require  material  solutions  and/or 
further  Research  and  Development  Test  and  Evaluation  have 
not  yet  been  addressed  and  are  repeated  below.  One  electronic 
warfare  recommendation  that  was  not  program-specific  is  also 
repeated  below. 

IDECM  System 

1.  The  Navy  should  restructure  and  reorganize  the  complex 
and  poorly  organized  IDECM  system  software  code. 

This  will  minimize  potential  software  problems  yet  to  be 
discovered  and  simplify  future  modifications. 

2.  The  Navy  should  develop  hardware  and/or  software 
changes  to  provide  pilots  with  correct  indications 
of  whether  a  decoy  was  completely  severed.  This 
recommendation  does  not  apply  to  the  F/A-18  C/D 
installation  since  that  installation  does  not  include  a  towed 
decoy. 

3.  The  Navy  should  investigate  the  effects  of  IDECM  on  threat 
missile  fuses. 

Electronic  Warfare  Warfighting  Improvements 

4.  In  coordination  with  the  Defense  Intelligence  Agency, 
the  Navy  should  update  the  threat  lethal  radii  and/or  the 
evaluation  processes  that  are  used  to  determine  whether 
simulated  shots  are  hits  or  misses. 

•  FY12  Recommendations.  None. 
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Joint  Standoff  Weapon  (JSOW) 


Executive  Summary 

•  The  Navy  completed  developmental  testing  and  initiated 
integrated  testing  of  the  AGM-154C-1  Joint  Standoff  Weapon 
(JSOW)  variant  during  FY12.  The  JSOW  C-l  integrated 
testing  completed  in  early  FY13,  with  operational  testing  to 
begin  in  mid-FY13. 

•  Preliminary  results  to  date  indicate  that  weapon  impact 
accuracy  for  moving  maritime  targets  is  well  within  the 
accuracy  requirement  value,  and  accuracy  performance  against 
stationary  land  targets  has  been  maintained. 

•  Preliminary  results  to  date  indicate  that  the  JSOW  C-l 
Mean  Flight  Hours  Between  Operational  Mission  Failure 
(MFHBOMF)  is  well  below  the  requirement  value,  primarily 
the  result  of  software-driven  problems.  Another  concern  is  the 
excessively  complicated  pilot-vehicle  interface  (PVI),  which 
could  prevent  successful  mission  execution. 

•  Planned  updates  to  the  software  to  address  these  problems 
may  invalidate  use  of  some  integrated  test  data  for  DOT&E’s 
operational  evaluation  of  JSOW  C-l. 

System 

•  The  AGM-154  JSOW  family  uses  a  common  and  modular 
weapon  body  capable  of  carrying  various  payloads.  The 
JSOW  is  a  1,000-pound  class,  air-to-surface  glide  bomb 
intended  to  provide  low  observable,  standoff  precision 
engagement  with  launch  and  leave  capability.  All  variants 
employ  a  tightly  coupled  GPS/Inertial  Navigation  System. 

•  AGM-154  A  (JSOW  A)  payload  consists  of  145  BLU-97/B 
combined  effects  submunitions. 

•  AGM-154C  (JSOW  C)  utilizes  an  imaging  infrared  seeker 
and  its  payload  consists  of  an  augmenting  charge  and 

a  follow-through  bomb  that  can  be  set  to  detonate  both 
warheads  simultaneously  or  sequentially. 

•  AGM-154A  and  AGM-154C  are  fielded  weapons,  and  are  no 
longer  under  DOT&E  oversight.  AGM-154C-1  (JSOW  C-l) 
adds  moving  maritime  target  capability  and  the  two-way  strike 
common  weapon  datalink  to  the  baseline  AGM-154C  weapon. 


Mission 

•  Combatant  Commanders  use  JSOW  A  to  conduct  pre-planned 
attacks  on  soft  point  and  area  targets  such  as  air  defense  sites, 
parked  aircraft,  airfield  and  port  facilities,  command  and 
control  antennas,  stationary  light  vehicles,  trucks,  artillery,  and 
refinery  components. 

•  Combatant  Commanders  use  JSOW  C  to  conduct  pre-planned 
attacks  on  point  targets  vulnerable  to  blast  and  fragmentation 
effects  and  point  targets  vulnerable  to  penetration  such  as 
industrial  facilities,  logistical  systems,  and  hardened  facilities. 

•  Units  will  use  JSOW  C-l  to  conduct  attacks  against  moving 
maritime  targets  and  have  the  ability  to  retarget  weapons 
post-launch.  JSOW  C-l  will  retain  the  JSOW  C  legacy 
capability  against  stationary  land  targets. 

Major  Contractor 

Raytheon  Company,  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  The  Navy  conducted  developmental  and  integrated  testing 
in  accordance  with  a  DOT&E-approved  Test  and  Evaluation 
Master  Plan  for  the  JSOW  C-l. 

•  The  Navy  completed  the  developmental  test  phase  in  FY12 
with  the  release  of  the  second  of  two  planned  free  flight 
weapon  drops  against  moving  maritime  targets. 

•  The  Navy  completed  the  four  planned  integrated  test  free 
flight  weapon  drops  in  4QFY12  through  1QFY13.  Two  were 


against  a  moving  maritime  target  and  two  regression  tests  of 
JSOW  C  legacy  capability  against  a  stationary  land  target. 

•  Results  from  the  developmental  and  integrated  testing  will 
support  an  Operational  Test  Readiness  Review  (OTRR)  in 
2QFY13.  The  Navy  has  scheduled  JSOW  C-l  operational 
testing  for  FY13  following  the  OTRR. 
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Assessment 

•  Navy  testing  of  JSOW  C-l  is  ongoing.  Preliminary  results  to 
date  indicate: 

-  Weapon  impact  accuracy  for  moving  maritime  targets  is 
well  within  the  accuracy  requirement  value  and  accuracy 
performance  against  stationary  land  targets  has  been 
maintained. 

-  JSOW  C-l  MFHBOMF  is  well  below  the  requirement 
value.  This  is  primarily  the  result  of  software-driven 
problems.  Achieving  adequate  assessment  of  MFHBOMF 
during  operational  testing  is  an  area  of  high  risk. 

-  Excessively  complicated  PVI  that  could  prevent  successful 
mission  execution  is  an  area  of  high  risk  during  operational 
testing. 

•  Planned  updates  to  the  JSOW  software  to  address  these 
problems  may  invalidate  use  of  some  developmental  and 


integrated  test  data  for  DOT&E’s  operational  evaluation  of 
JSOW  C-l. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
satisfactorily  addressed  previous  recommendations  for 
JSOW  A  and  C.  There  are  no  previous  recommendations  for 
JSOW  C-l  since  it  is  a  new  variant  of  the  AGM-154. 

•  FY12  Recommendation. 

1.  Before  proceeding  to  JSOW  C-l  operational  testing,  the 
Navy  should  verify  that  newly  incorporated  software 
updates  adequately  reduce  software-driven  failures  and  that 
PVI  complexity  have  been  mitigated  sufficiently  to  permit 
successful  mission  execution. 
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Light  Armored  Vehicle  (LAV)  Upgrade 


Executive  Summary 

•  The  Marine  Corps  has  developed  a  special  purpose  kit  to 
improve  protection  from  under-vehicle  attacks.  This  kit 
(known  as  the  D-Kit)  is  designed  to  work  with  the  Ballistic 
Protection  Upgrade  Package  (BPUP)  and  is  installed  at  the 
discretion  of  the  operational  commander. 

•  The  Marine  Corps  completed  eight  system-level  underbody 
blast  tests  in  March  2012  at  Aberdeen  Test  Center,  Maryland; 
the  data  indicate  that  the  D-kit  increases  crew  protection. 

System 

•  The  Family  of  Light  Armored  Vehicles  (LAV)  shares  a 
common  base  platform  configuration  (eight-wheels,  armored 
hull,  suspension,  power  plant,  drive  train,  and  auxiliary 
automotive  subsystem)  among  eight  Mission  Role  Variants 
(MRVs).  The  LAV-25  personnel  carrier  is  the  predominant 
MRV. 

•  The  Marine  Corps  initiated  a  Service  Life  Extension  Program 
in  FY05  primarily  to  address  obsolescence  deficiencies.  The 
Marine  Corps  undertook  the  Survivability  Upgrade  I  program 
based  on  an  Urgent  Need  Statement  from  the  operating  forces. 
This  upgrade  became  the  LAV  A2  configuration  standard, 
and  involved  developing  and  installing  a  BPUP,  power  pack 
enhancements,  upgraded  suspension,  and  other  modifications. 

•  The  BPUP  system  consists  of  three  kits,  two  of  which  provide 
additional  protection  against  threats,  while  the  third  provides 
an  internal  and  external  stowage  system. 

•  In  2007,  the  Program  Management  LAV  Office  internally 
designed  an  underbody  kit  (known  as  a  D-Kit)  that  can  be 
incorporated  to  counteract  under-vehicle  strikes.  The  D-kit 
has  been  fielded  since  2009. 

•  The  LAV  A2  D-Kit  is  designed  to  work  with  the  previously 
installed  BPUP  system  and  is  a  special  purpose  mission  kit 
used  in  theater  at  the  discretion  of  the  operational  commander. 


LAV-25  A2  Variant 


The  BPUP  provides  armor  protection  to  the  sides  and  front 
of  the  vehicle,  whereas  the  D-Kit  provides  additional  armor 
protection  with  a  V-shaped  hull  attachment  under  the  vehicle. 

Mission 

Marine  Corps  commanders  will  use  LAVs  to  provide  combined 

arms  reconnaissance,  security  missions,  and  mobile  electronic 

support. 

Major  Contractors 

•  General  Dynamics  Land  Systems  -  Canada 

•  Conversion  of  a  LAV  A1  to  a  LAV  A2  is  conducted  at  Marine 
Corps  Logistics  Base  -  Albany,  Georgia,  and  Marine  Corps 
Logistics  Base  -Barstow,  California 


Activity 

•  Follow-on  System-Level  Underbody  Testing  completed  in 
March  2012  at  Aberdeen  Test  Center,  Maryland,  in  accordance 
with  the  DOT&E-approved  LFT&E  Strategy  and  the  Event 
Design  Plan.  The  LAV  Program  Office  provided  two 
fully-armored  LAV-25  A2  assets  to  test  and  characterize  the 
force  protection  capabilities  and  vehicle  vulnerability  against 
underbody  blast  threats.  The  test  also  included  Mine  Resistant 
Ambush  Protected  All-Terrain  Vehicle  level  threats,  as  well  as 
threshold  threats,  and  a  single  test  without  the  D-Kit. 

•  DOT&E  will  provide  a  classified  LFT&E  report  to  Congress. 


Assessment 

•  Analysis  of  data  after  completion  of  the  Follow-on  System 
Level  Underbody  Testing  confirms  DOT&E’s  assessment  of 
emerging  results  from  FY 1 1 . 

•  Testing  and  analysis  confirm  that  the  LAV-25  A2  D-Kit 
increases  crew  protection  against  some  under-vehicle  mine  and 
IED  strikes.  The  details  are  available  in  the  December  2012 
classified  DOT&E  LFT&E  report. 

•  The  location  of  the  LAV-25  A2  fuel  cell,  which  is  centered 
under  the  rear  of  the  vehicle,  increases  crew  vulnerability  to 
some  under- vehicle  threats. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Marine  Corps  is 
addressing  the  previous  two  recommendations  through  funding 
efforts  in  the  Program  Objective  Memorandum  for  FY14,  and 
is  pursuing  additional  LAV  survivability  upgrades  (such  as 
blast  mitigation  seats,  5-point  harness  seat  belts,  and  advanced 
suspension  designs)  with  development  and  procurement  slated 
for  the  FY 17-20  timeframe.  Additionally,  the  Marine  Corps  is 
actively  engaged  in  the  recommendation  to  consider  relocating 


the  fuel  cell  of  the  LAV-25  A2,  by  utilizing  the  LAV  MRV 
fuel  cell  relocation  program  as  a  pre-cursor  to  a  LAV-25  A2 
fuel  cell  relocation  program.  The  results  of  the  MRV  fuel 
cell  relocation  program  will  aid  the  program  manager  with 
engineering  analysis  for  the  subsequent  LAV-25  A2  fuel  cell 
relocation. 

•  FY12  Recommendations.  None. 
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Littoral  Combat  Ship  (LCS) 


Executive  Summary 

•  The  draft  revision  of  the  Littoral  Combat  Ship  (LCS)  Test  and 
Evaluation  Master  Plan  (TEMP)  currently  being  proposed  by 
the  Navy  will  require  additional  revisions  to  be  approved  by 
DOT&E.  In  particular,  the  TEMP  must  incorporate  phased 
operational  testing  of  all  increments  of  mission  module 
capability  to  be  deployed  for  use  in  combat. 

•  The  Navy  commenced  a  Quick  Reaction  Assessment  (QRA) 
of  the  gun  systems  on  LCS  1  in  June  2012.  Results  from  the 
QRA  revealed  performance,  reliability,  and  operator  training 
deficiencies  for  both  the  30  mm  and  57  mm  guns. 

•  The  Navy  conducted  testing  of  the  MH-60S  Block  2  Airborne 
Mine  Countermeasures  (AMCM)  System,  which  is  intended 
to  support  LCS  mine  countermeasures.  Testing  indicated 
shortfalls  in  performance: 

-  The  Navy  determined  the  MH-60S  helicopter  cannot 
safely  tow  the  AN/AQS-20A  Sonar  Mine  Detecting  Set 
(AQS-20A)  or  the  Organic  Airborne  Sweep  and  Influence 
System  (OASIS)  because  the  helicopter  is  underpowered 
for  these  operations.  The  MH-60S  helicopter  will  no 
longer  be  assigned  these  missions  operating  from  any  ship, 
including  LCS. 

-  Preliminary  evaluation  of  test  data  collected  during 
operational  assessment  (OA)  of  the  MH-60S  Block  2 
Airborne  Laser  Mine  Detection  System  (ALMDS) 
indicates  that  the  system  does  not  meet  Navy  requirements 
for  False  Classification  Density  and  has  low  reliability. 

•  DOT&E  agreed  to  defer  the  Total  Ship  Survivability  Trials 
(TSSTs)  from  LCS  1  and  2  to  LCS  3  and  4,  which  affords  the 
Navy  time  to  complete  pre-trial  damage  scenario  analysis. 

•  DOT&E  also  agreed  to  defer  the  Shock  Trials  from  LCS  3 
and  4  to  LCS  5  and  6,  resulting  in  a  one-year  delay,  due  to 
significant  seaframe  and  system  design  changes  expected. 

LCS  5  and  6  will  be  most  representative  of  the  class  for 
purposes  of  the  Shock  Trials. 

System 

•  The  LCS  is  designed  to  operate  in  the  shallow  waters  of  the 
littorals  where  larger  ships  cannot  maneuver  as  well.  It  is 
intended  to  accommodate  a  variety  of  individual  warfare 
systems  (mission  modules)  assembled  and  integrated  into 
interchangeable  mission  packages. 

•  The  Navy  currently  plans  to  field  Mine  Countermeasure 
(MCM),  Surface  Warfare  (SUW),  and  Anti-Submarine 
Warfare  (ASW)  mission  packages. 

•  The  Navy  is  procuring  two  ship  (seaframe)  variants: 

-  USS  Freedom  (LCS  1,  3,  5,  and  follow-on  ships)  is  a 
semi-planing  monohull  design  constructed  of  steel  (hull) 
and  aluminum  (deckhouse)  with  a  combined  diesel  and  gas 
turbine  main  propulsion  system. 


-  USS  Independence  (LCS  2,  4,  6,  and  follow-on  ships)  is 
an  aluminum  trimaran  design  driven  by  four  independent 
steerable  water  jets. 

•  Common  design  specifications: 

-  Sprint  speed  in  excess  of  40  knots,  draft  of  less  than 

20  feet,  and  unrefueled  range  in  excess  of  3,500  nautical 
miles  at  14  knots 

-  Accommodations  for  up  to  76  personnel  (air  detachment 
personnel,  mission  module  personnel,  and  a  core  crew  of 
no  more  than  40) 

-  A  Common  Mission  Package  Computing  Environment  for 
mission  package  control 

-  Hangars  sized  to  embark  MH-60R/S  and  Vertical  Take-Off 
Unmanned  Aerial  Vehicles  (VTUAVs) 

-  57  mm  Bofors  Mk  3  gun 

•  The  designs  have  different  core  combat  systems  to  provide 

command  and  control,  situational  awareness,  and  self  defense 

against  anti-ship  cruise  missiles  and  surface  craft. 

-  Freedom  Variant:  COMB ATSS-21,  an  Aegis-based 
integrated  combat  weapons  system  with  a  TRS-3D 
air/surface  search  radar,  Ship  Self-Defense  System  Rolling 
Airframe  Missile  (RAM)  system  (one  21 -cell  launcher), 
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and  a  DORNA  Electro-Optical/Infrared  system  for  Mk  110 
57  mm  gun  fire  control. 

-  Independence  Variant:  Integrated  combat  management 
system  (derived  from  Dutch  TACTICOS  system)  with  a 
Sea  Giraffe  air/surface  search  radar,  one  RAM  (11-cell) 
launcher  integrated  with  the  Close-In  Weapons  System 
(Mk  15)  search  and  fire  control  radars  (called  SeaRAM), 
and  Sea  Star  SAFIRE  Electro-Optical/Infrared  systems  for 
57  mm  gun  fire  control. 

•  Multiple  individual  programs  of  record  involving  sensor  and 
weapon  systems  and  off-board  vehicles  make  up  the  individual 
mission  modules.  Mission  modules  provide  offensive 
capability. 

-  SUW  Mission  Package: 

■  Gun  Mission  Modide  (two  Mk  46  30  mm  guns) 
(Increment  1) 

■  Aviation  Module  (embarked  MH-60R  and  VTUAV) 
(Increment  1) 

■  Maritime  Security  Modide  (small  boats)  (Increment  2) 

■  Surface-to-Surface  Missile  system  intended  to  provide 
limited  “interim”  SUW  capability  in  response  to  an 
urgent  operational  need  (Increment  3) 

■  Longer  range  Surface-to-Surface  Missile  (Increment  4) 

-  MCM  Mission  Package: 

■  Remote  Minehunting  System  (RMS),  consisting  of 
the  Remote  Multi-Mission  Vehicle  (RMMV)  and  the 
AQS-20A  sonar  system  (Increment  1) 

■  MH-60S  Block  2A/B  AMCM  System,  consisting 
of  an  AMCM  system  operator  workstation,  a  tether 
system,  and  the  two  MCM  systems  currently  under 
development  -  ALMDS  for  detection  and  classification 
of  near-surface  mines,  and  the  Airborne  Mine 
Neutralization  System  (AMNS)  for  identification  and 
neutralization  of  in-volume  and  bottom  mines  (the 
AQS-20A  sonar  system  and  OASIS  are  no  longer  being 
developed  for  use  in  the  AMCM  System)  (Increment  1) 

■  AMNS  Pre-Planned  Product  Improvement  (P3I) 

Program  for  neutralization  of  near-surface  mines  and 
Coastal  Battlefield  Reconnaissance  and  Analysis  Block  I 
(COBRA  Blk  I)  system  for  unmanned  aerial  tactical 
reconnaissance  to  detect  and  localize  minelines  and 
obstacles  in  the  daylight  in  the  beach  zone  and  partially 
in  the  surf  zone  (Increment  2) 

■  Unmanned  Influence  Sweep  System  (UISS)  to 
activate  acoustic-,  magnetic-,  and  combined 
acoustic/ magnetic-initiated  volume  and  bottom  mines  in 
shallow  water  so  they  self-destruct  (Increment  3) 

■  COBRA  Block  II  system,  which  has  Block  I  capability 
with  the  addition  of  night-time  minefield  and  obstacle 


detection  capability  and  full  detection  capability  in 
surf  zone;  and  Knifefish  Unmanned  Undersea  Vehicle, 
a  self-propelled,  untethered,  autonomous  underwater 
vehicle,  employing  a  low-frequency  broadband  sonar 
sensor  to  detect,  classify,  and  identify  volume  and  bottom 
mines  in  shallow  water  (Increment  4) 

-  ASW  Mission  Package: 

■  Torpedo  Defense  and  Countermeasures  Module  (Light 
Weight  Tow  torpedo  countermeasure)  (Increment  2) 

■  ASW  Escort  Module  (Multi-Function  Towed  Array  and 
Variable  Depth  Sonar)  (Increment  2) 

■  Aviation  Module  (embarked  MFI-60R  and  two  VTUAVs) 
(Increment  2) 

•  The  Navy  plans  to  acquire  a  total  of  55  LCSs.  In  early  FY11, 
the  USD(AT&L)  authorized  the  procurement  of  hulls  3 
through  22  (10  of  each  ship  design),  subject  to  Congressional 
appropriations. 

Mission 

•  The  Maritime  Component  Commander  will  employ  LCS  to 
conduct  MCM,  ASW,  or  SUW  tasks  depending  on  the  mission 
package  fitted  into  the  seaframe.  With  the  Maritime  Security 
Module,  installed  as  part  of  the  SUW  mission  package,  the 
ship  can  conduct  Visit,  Board,  Search,  and  Seizure  maritime 
interception  operations.  Commanders  can  employ  LCS  in 

a  maritime  presence  role  in  any  configuration  because  of 
capabilities  inherent  to  the  seaframe. 

•  The  Navy  can  employ  LCS  alone  or  in  company  with  other 
ships.  The  Navy  is  still  developing  the  concept  of  employment 
and  operations  for  these  ships  in  each  of  the  mission  areas. 

Major  Contractors 

•  Freedom  Variant  (LCS  1,  3,  5,  7,  and  follow-on  ships) 

-  Prime:  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Washington,  District  of  Columbia 

-  Shipbuilder:  Marinette  Marine  -  Marinette,  Wisconsin 

•  Independence  Variant  (LCS  2,  4,  6,  8,  and  follow-on  ships) 

-  Prime  for  LCS  2  and  LCS  4:  General  Dynamics 
Corporation  Marine  Systems,  Bath  Iron  Works  -  Bath, 
Maine 

-  Prime  for  LCS  6,  LCS  8,  and  follow-on  ships:  Austal 
USA  -  Mobile,  Alabama 

-  Shipbuilder:  Austal  USA- Mobile,  Alabama 

•  Mission  Packages 

-  Future  Mission  Package  Integration  contract  awarded  to 
Northrop  Grumman  -  Los  Angeles,  California 
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Activity 

Seaframe 

•  Freedom  Variant  (LCS  1): 

-  The  Navy  completed  the  first  phase  of  the 
Post-Shakedown  Availability  (PSA),  which  commenced 
in  3QFY11,  on  LCS  1.  During  sea  trials  following 
this  event,  the  ship  developed  a  shaft  seal  leak  and 
subsequently  reentered  dry-dock  for  six  weeks  to  repair. 

-  The  Navy’s  Board  of  Inspection  and  Survey  (INSURV) 
graded  LCS  1  as  fit  for  service  during  special  trials  in 
May  2012  following  the  emergent  dry-docking. 

-  The  Navy  continued  developmental  testing  of  the  57  mm 
gun  system  on  LCS  1 . 

-  The  Navy’s  Commander,  Operational  Test  and 
Evaluation  Force  (COTF)  commenced  a  QRA  on  LCS  1 
in  support  of  FY13  early  deployment.  Part  1  (57  mm 
gun  assessment)  of  the  QRA  began  in  3QFY12.  Part  II 
(Information  Assurance  and  Maritime  Interdiction 
Operations  assessments)  will  take  place  on  LCS  1  during 
1QFY13. 

-  The  Navy  started  the  second  PSA  in  July  2012  on  LCS  1 
in  San  Diego,  California. 

•  Independence  Variant  (LCS  2): 

-  The  Navy  commissioned  LCS  2  in  January  2010  and 
began  MCM  mission  module  developmental  testing  in 
Mobile,  Alabama,  after  commissioning. 

-  LCS  2  departed  the  east  coast  and  arrived  in  San  Diego, 
California,  in  May  2012. 

-  The  Navy  commenced  the  first  phase  of  the  nine-month 
PSA  in  September  2012  in  San  Diego,  California. 

•  Freedom  Variant  (LCS  3): 

-  INSURV  evaluated  LCS  3  as  satisfactory  during 
acceptance  trials  in  April  2012. 

-  The  Navy  commissioned  LCS  3  in  Galveston,  Texas,  in 
September  2012. 

SUW  Module 

•  COTF  tested  the  30  mm  gun  on  LCS  1  during  the 
QRA  in  June  and  July  2012  in  accordance  with  the 
DOT&E-approved  data  collection  plan.  The  Navy 
continued  developmental  testing  of  the  30  mm  gun  system 
on  LCS  1. 

MCM  Module 

•  The  Navy  conducted  testing  of  the  MH-60S  Block  2 
AMCM  System,  which  is  intended  to  support  LCS  MCM. 

-  COTF  completed  testing  of  the  MH-60S  Block  2A 
AMCM  System  with  the  AQS-20A  sonar  system  in 
4QFY11  in  accordance  with  the  DOT&E-approved  test 
plan.  DOT&E  issued  an  OA  report  in  June  2012. 

-  COTF  commenced  Phase  A  (Shore-based  and  Training 
Phase)  of  the  planned  OA  of  the  MH-60S  Block  2 
AMCM  System  with  the  ALMDS  in  2QFY12;  testing 
completed  in  4QFY12,  and  was  conducted  in  accordance 
with  the  DOT&E-approved  test  plan.  DOT&E  expects 
to  issue  a  formal  test  report  in  2QFY13.  The  Navy 
postponed  conduct  of  Phase  B  (LCS  Ship-based  Phase) 
of  the  planned  OA  due  to  the  unavailability  of  an 


LCS  seaframe  to  facilitate  conduct  of  MCM  mission 
module  testing.  The  Navy  intends  to  conduct  the  LCS 
ship-based  phase  of  the  planned  ALMDS  and  AMNS 
OAs  in  conjunction  with  the  LCS  Technical  Evaluation 
scheduled  to  occur  in  FY14. 

•  The  RMS  program  completed  reliability  growth  testing 
(developmental  testing)  of  RMMV  version  4.1  in  1QFY12. 

•  The  Navy  commenced  a  ship-based  phase  of  MCM  mission 
module  developmental  testing  (DT-B2)  in  1QFY12;  testing 
completed  in  4QFY12. 

LFT&E 

•  The  Navy  revised  the  survivability  requirements  for  LCS  3 
and  beyond  to  describe  the  ships’  survivability  requirements 
in  terms  of  class-specific  LCS  Vulnerability  Levels: 

-  LCS  Vulnerability  Level  1  -  Operate  emergency  and 
damage  control  sys terns/ equipment  to  provide  for  an 
orderly  abandon  ship. 

-  LCS  Vulnerability  Level  II  -  All  of  the  capabilities 
of  LCS  Vulnerability  Level  I,  plus  the  capability  for 
mobility  to  exit  the  immediate  area,  electrical  power  and 
other  required  services  to  operate  vital  systems,  exterior 
communications  to  support  contact  with  the  operational 
commander,  and  small-to-medium  caliber  weapons  or 
equivalent  capability  to  prevent  boarding  from  small 
craft. 

-  LCS  Vulnerability  Level  III  -  All  of  the  capabilities  of 
LCS  Vulnerability  Level  II,  plus  retain  some  critical 
mission  capability  as  defined  in  Conditions  for  Total 
Ship  Survivability  Analyses,  Test,  and  Evaluation  for 
Susceptibility  and  Vulnerability/Recoverability. 

•  DOT&E  agreed  to  defer  the  TSST  from  LCS  1  and  2  to 
LCS  3  and  4.  This  delay  affords  the  Navy  enough  time  to 
complete  the  needed  pre-trial  damage  scenario  analyses. 

The  TSST  is  currently  scheduled  to  be  conducted  on  LCS  3 
in  December  2013  and  on  LCS  4  in  August  2014. 

•  DOT&E  also  agreed  to  defer  the  Shock  Trials  from  LCS  3 
and  4  to  LCS  5  and  6,  resulting  in  a  one-year  delay.  With 
significant  seaframe  and  system  design  changes  expected, 
LCS  5  and  6  will  be  most  representative  of  the  respective 
class  for  purposes  of  Shock  Trials.  LCS  5  and  6  will  also 
be  the  first  ships  to  include  shock-qualified  equipment. 

•  DOT&E  reviewed  drafts  of  the  Navy’s  Detail  Design 
Integrated  Survivability  Assessment  Reports  for  LCS  1  and 
2.  The  Navy  is  working  to  address  DOT&E’s  comments 
and  finalize  these  reports. 

•  The  Navy  is  planning  surrogate  tests  to  address  knowledge 
gaps  related  to  the  vulnerability  of  an  aluminum  ship 
structure  to  weapon-induced  blast  and  fire  damage.  These 
tests  will  be  conducted  during  FY13  and  FY14. 

•  DOT&E  approved  the  57  mm  ammunition  LFT&E 
Management  Plan,  which  details  the  test  and  evaluation 
necessary  to  evaluate  the  lethality  of  the  57  mm 
ammunition.  The  Navy  is  coordinating  with  the  Finnish 
Navy  to  use  their  operational  equipment  to  conduct  an 
effectiveness  test  exercise  in  September  2013. 
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•  DOT&E  approved  the  30  mm  ammunition  LFT&E 
Management  Plan,  which  details  the  test  and  evaluation 
necessary  to  evaluate  the  lethality  of  the  30  mm 
ammunition. 

•  The  Navy  will  submit  the  Surface-to-Surface  Missile 
LFT&E  Management  Plan,  which  details  the  test  and 
evaluation  necessary  to  evaluate  the  lethality  of  the  missile, 
for  approval  in  FY13. 

•  Fire  insulation  testing  was  successfully  conducted  on  a 
flight  deck  section  of  LCS  2  (referred  to  as  grillage  test) 
on  the  Ex-USS  Shadwell  in  March  2012.  The  test  article 
exceeded  performance  requirements  for  a  fully  intact 
insulated  flight  deck.  Weapon-induced  blast  and  fire 
damage  will  be  addressed  in  surrogate  testing  planned  for 
FY13  and  FY14. 

Assessment 

This  assessment  is  based  on  information  from  DOT&E’s 
observations  of  selected  events  and  operations.  The  program 
offices  have  issued  limited  developmental  test  results  and  have 
not  been  able  to  provide  developmental  test  data  for  independent 
analysis.  No  formal  at-sea  operational  tests  were  conducted. 

Program 

•  The  draft  revision  of  the  LCS  TEMP  currently  being 
proposed  by  the  Navy  will  require  additional  revisions  to 
be  approved  by  DOT&E.  In  particular,  the  TEMP  must 
incorporate  phased  operational  testing  of  all  increments  of 
mission  module  capability  to  be  deployed  for  use  in  combat. 

Seaframe 

•  Freedom  Variant  (LCS  1  and  3): 

-  As  reported  in  the  FY 1 1  Annual  Report,  the  Navy 
discovered  cracks  in  the  hull  and  superstructure  of  LCS  1 
that  required  interim  repairs  as  well  as  design  changes. 
The  Navy  made  production  changes  to  reduce  cracking 
on  LCS  3;  cracking  has  not  been  observed  to  date. 

•  Independence  Variant  (LCS  2): 

-  As  noted  in  the  FY  1 1  Annual  Report,  the  Navy 
completed  interim  repairs  on  LCS  2  because  of 
aggressive  galvanic  corrosion  in  the  vicinity  of  water 
jets.  The  Navy  is  installing  a  system  to  prevent  corrosion 
on  LCS  2  during  the  current  PSA.  An  Impressed  Current 
Cathodic  Protection  system  is  planned  for  the  water  jet 
tunnels  on  LCS  4. 

-  The  Navy  continued  to  work  through  problems 
associated  with  the  Twin  Boom  Extensible  Crane 
on  LCS  2.  Limited  testing  to  date  precludes  further 
assessment  of  this  variant. 

•  Both  variants: 

-  Crew  size  can  limit  the  mission  capabilities  of  the  ship. 
Core  crew  size  provides  little  flexibility  to  support 
more  than  one  operation  at  a  time;  unplanned  manning 
losses  and  corrective  maintenance  further  exacerbate  the 
problem.  The  Navy  is  reviewing  manning  levels  and 
installing  20  additional  bunks  in  LCS  1  for  flexibility 
during  its  deployment,  but  is  not  changing  the  final 
manning  levels. 


-  Ship  operations  at  high  speeds  cause  vibrations  that 
make  accurate  use  of  the  57  mm  gun  very  difficult. 
Insufficient  operator  training  and  proficiency  also  appear 
to  have  contributed  to  the  poor  performance  of  the 

57  mm  gun. 

SUW  Module 

•  Both  variants: 

-  The  Navy  has  not  finalized  how  the  ships  will  be  utilized 
with  the  SUW  mission  module.  Additionally,  the  Navy 
has  not  completed  the  revised  capabilities  document 
defining  the  incremental  approach  to  fielding  mission 
modules. 

-  The  30  mm  guns  and  associated  combat  system  exhibit 
reliability  problems.  The  Navy  established  a  Failure 
Review  Board  to  identify  and  correct  deficiencies  in 
30  mm  gun  performance. 

•  Freedom  Variant:  Performance  deficiencies  with 
COMBATSS-21  and  TRS-3D  affect  tracking  and 
engagement  of  contacts. 

MCM  Module 

•  Testing  of  the  MH-60S  Block  2  AMCM  System  revealed 
significant  shortfalls  in  performance. 

-  The  MH-60S  helicopter  with  the  AQS-20A  sonar  is  not 
operationally  effective  or  suitable  because  the  helicopter 
is  underpowered  and  cannot  safely  tow  the  sonar  under 
the  variety  of  conditions  necessary.  The  Navy  cancelled 
the  MH-60S  helicopter  mission  to  tow  the  AQS-20A  and 
OASIS.  The  cancellation  of  the  OASIS  mission  creates 
a  gap  in  LCS  organic  mine  sweeping  capability  that 

the  Navy  intends  to  address  with  the  implementation  of 
UISS  in  Increment  3  of  the  MCM  mission  module. 

-  As  observed  during  the  OA  and  developmental  testing, 
the  AQS-20A  does  not  meet  some  Navy  requirements. 
Contact  depth  localization  errors  exceeded  Navy  limits 
in  all  AQS-20A  operating  modes.  False  contacts  also 
exceeded  Navy  limits  in  two  of  three  search  modes. 

The  Navy  has  implemented  modified  tactics  intended 
to  mitigate  these  deficiencies;  however,  those  tactics 
limit  platform-level  productivity  (Area  Coverage  Rate 
Sustained).  Additionally,  the  Navy  is  developing  a  P3I 
program  to  correct  these  deficiencies. 

-  The  analysis  of  test  data  collected  during  Phase  A 
of  the  OA  of  the  MH-60S  and  ALMDS  is  still  in 
progress.  Preliminary  evaluation  of  data  collected 
during  the  OA  suggests  that  the  ALMDS  does  not  meet 
Navy  requirements  for  False  Classification  Density  or 
reliability.  DOT&E  expects  to  issue  a  formal  test  report 
in  2QFY13.  The  Navy  has  implemented  modified  tactics 
intended  to  mitigate  this  deficiency;  however,  those 
tactics  limit  platform-level  productivity  (Area  Coverage 
Rate  Sustained).  Additionally,  the  Navy  is  developing  a 
P3I  program  to  correct  this  deficiency.  Phase  B  testing 
was  originally  intended  to  provide  early  operational 
testing  insight  into  the  operational  effectiveness  and 
suitability  of  AMCM  systems  when  operating  from  an 
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LCS,  and  to  identify  risk  to  the  successful  completion  of 
IOT&E.  However,  the  Navy’s  postponement  of  Phase  B 
testing  will  eliminate  these  intended  benefits. 

•  As  reported  by  the  Navy,  the  reliability  of  RMMV 
version  4.1  grew  as  predicted  by  the  program’s  reliability 
growth  curve.  However,  the  observed  growth  is  predicated 
on  limited  test  data  collected  in  a  minimally  stressing 
operational  environment.  The  limited  scope  of  testing 
prevents  any  meaningful  conclusions  about  operational 
availability  of  the  RMS. 

•  As  observed  and  reported  by  the  Navy,  during 
developmental  testing  (DT-B2),  launch  and  recovery  of 
the  RMS  was  problematic  due  to  material  deficiencies 
with  launch  and  recovery  systems,  manpower  and 
training  deficiencies,  and  compatibility  with  the  operating 
environment. 

LFT&E 

•  LCS  is  not  expected  to  be  survivable  in  that  it  is  not 
expected  to  maintain  mission  capability  after  taking 
a  significant  hit  in  a  hostile  combat  environment. 

This  assessment  is  based  on  a  review  of  LCS  design 
requirements,  which  do  not  require  the  inclusion  of  the 
survivability  features  necessary  to  conduct  sustained 
operations  in  its  expected  combat  environment.  DOT&E’s 
review  of  the  Navy’s  draft  Detail  Design  Integrated 
Survivability  Assessment  Reports  has  not  changed  this 
assessment. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  partially 
addressed  one  FY09  recommendation  to  develop  an  LFT&E 
program  with  the  approval  of  the  LFT&E  Management  Plan; 
however,  the  recommendation  will  not  be  fully  addressed 
until  the  details  of  the  surrogate  testing  and  the  lethality 


testing  are  developed.  The  Navy  has  partially  addressed  the 
FY10  recommendations  to  implement  all  recommendations 
from  DOT&E’s  Combined  Operational  and  Live  Fire  Early 
Fielding  Report.  Significant  remaining  recommendations 
from  the  Early  Fielding  Report  include  enhancing  sensors  and 
improving  capability  of  gun  systems.  With  respect  to  FY 1 1 
recommendations,  the  Navy  is  adjusting  tactics  and  increasing 
funding  to  address  deficiencies  with  the  AQS-20A  and 
ALMDS.  The  FY11  recommendation  for  the  Navy  to  continue 
to  report  vulnerabilities  during  live  fire  tests  remains  valid. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Complete  the  revised  capabilities  document  defining  the 
incremental  approach  to  fielding  mission  modules. 

2.  Publish  the  concept  of  operations  for  all  the  mission 
modules. 

3.  Complete  manning  level  studies  and  finalize  manning  prior 
to  LCS  IOT&E. 

4.  Correct  gun  reliability  issues  identified  during  QRA.  These 
problems  need  to  be  addressed  prior  to  completion  of  the 
LCS  SUW  Mission  Package  IOT&E. 

5.  Conduct  LCS  ship-based  phases  of  the  planned  OA  of  the 
MH-60S  Block  2  and  ALMDS  as  well  as  an  OA  of  the 
MH-60S  Block  2  and  AMNS  MCM  systems  in  FY13  to 
reduce  risk  to  the  LCS  MCM  Mission  Package  IOT&E. 

6.  Investigate  and  correct  material  deficiencies  with  mission 
module  launch  and  recovery  systems,  and  manpower 
and  training  deficiencies  that  prevent  safe  and  effective 
shipboard  launch  and  recovery  of  the  RMS. 
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LPD-17  San  Antonio  Class  Amphibious  Transport  Dock 


Executive  Summary 

•  The  Navy  demonstrated  the  efficacy  of  LPD-17’s  collective 
protection  system,  its  countermeasure  wash-down 
system,  and  the  crew’s  ability  to  decontaminate  the 

ship’s  equipment,  personnel,  and  spaces,  and  to  conduct 
amphibious  operations  in  chemical  protective  suits  during 
FOT&E  in  February  2012. 

•  The  Navy  conducted  FOT&E  in  August  2012  to  examine 
the  reliability  of  systems  during  the  first  five  hours  of 

an  amphibious  assault.  Testing  was  not  sufficient  to 
demonstrate  that  the  ship  class  satisfies  its  requirements. 

•  The  Navy  is  working  to  correct  deficiencies  identified 
during  IOT&E  that  led  DOT&E  to  assess  the  ship  not 
operationally  effective,  not  operationally  suitable,  and  not 
survivable  in  a  hostile  environment.  Flowever,  correction 
of  a  number  of  these  deficiencies  has  not  yet  been  verified 
by  follow-on  operational  testing. 

•  The  Navy’s  Board  of  Inspection  and  Survey  (INSURV) 
assessed  the  material  condition  of  LPD-17,  LPD-22,  and 
LPD-23  as  satisfactory. 

System 

•  LPD-17  is  a  diesel  engine -powered  ship  designed 
to  embark,  transport,  and  deploy  ground  troops  and 
equipment.  Ship-to-shore  movement  is  provided  by 
Landing  Craft  Air  Cushion  (LCAC),  Landing  Craft  Utility 
(LCU),  Amphibious  Assault  Vehicles  (AAVs),  MV-22 
tiltrotor  aircraft,  and/or  helicopters.  Key  ship  features  and 
systems  include  the  following: 

-  A  floodable  well  deck  for  LCAC,  LCU,  and  AAV 
operations 

-  A  flight  deck  and  hangar  to  support  Navy  and  Marine 
Corps  aircraft  and  helicopters 

-  Command,  Control,  Communications,  Computers,  and 
Intelligence  facilities  and  equipment  to  support  Marine 
Corps  Landing  Force  operations 

-  Self  defense  against  anti-ship  cruise  missile  capability 
provided  by  the  Ship  Self-Defense  System  (SSDS) 

Mk  2-based  combat  system,  which  includes  the 
Cooperative  Engagement  Capability  radar  tracking 
system  and  data  distribution  system,  the  Rolling 
Airframe  Missile  point  defense  system,  the  SLQ-32B 
(V)2  (with  Mk  53  Decoy  Launching  System  with  Nulka 
electronic  decoys)  passive  electronic  warfare  system, 
and  radars  (SPQ-9B  horizon  search  radar  and  SPS-48E 
long-range  air  search  radar) 


-  Two  Mk  46  30  mm  gun  systems  and  smaller  caliber 
weapons  (e.g.,  Mk  2  .50-caliber  machine  guns)  to 
provide  the  ship’s  self-defense  against  small  surface 
threats 

-  A  Shipboard  Wide  Area  Network  that  serves  as  the 
data  backbone  for  most  of  the  ship’s  computer  systems 
(LPD- 1 7  is  one  of  the  first  ships  built  with  a  fully 
integrated  data  network  system). 

-  Design  features  that  reduce  the  ship’s  radar  cross 
section  and  are  intended  to  make  the  ship  less 
susceptible  to  attack 

Mission 

•  A  Fleet  Commander  will  employ  LPD-17  class  ships  as 

part  of  a  notional,  three-ship  Amphibious  Ready  Group  or 

independently  to  conduct  Amphibious  Warfare. 

•  The  Commanding  Officer  will  use  these  ships  to: 

-  Transport  combat  and  support  elements  of  a  Marine 
Expeditionary  Unit  or  Brigade 

-  Embark,  launch,  and  recover  LCACs,  LCUs,  and  AAVs 
for  amphibious  assault  missions 

-  Support  aerial  assaults  by  embarking,  launching,  and 
recovering  Marine  Corps  aircraft 

-  Carry  and  discharge  cargo  to  sustain  the  landing  force 

-  Conduct  non-combatant  evacuation  operations  and 
other  crisis  response  missions 

Major  Contractor 

Huntington  Ingalls  Industries  -  Pascagoula,  Mississippi 


LPD-17 


169 


NAVY  PROGRAMS 


Activity 

•  The  Navy  completed  FOT&E  designed  to  demonstrate 
capability  to  conduct  operations  in  a  chemical  warfare 
environment  (onboard  LPD-21)  in  February  2012. 

•  The  Navy  conducted  limited  reliability  FOT&E  on  ship 
systems  during  the  first  five  hours  of  an  amphibious  assault 
(onboard  LPD-20)  in  August  2012. 

•  The  Navy’s  INSURV  assessed  the  material  condition 
of  LPD-17,  LPD-22,  and  LPD-23  as  satisfactory  during 
Acceptance  Trials. 

•  The  Navy,  using  the  Probability  of  Raid  Annihilation  Test 
Bed  Modeling  and  Simulation  tool,  commenced  a  modeling 
and  simulation  study  to  determine  if  upgrades  and  corrections 
to  the  ship’s  combat  system  improve  the  ship’s  capability  to 
defeat  raids  of  anti-ship  cruise  missiles. 

•  The  Navy  provided  its  final  survivability  assessment 
identifying  99  deficiencies  to  DOT&E  in  FY12  and 
commenced  corrective  actions.  Flowever,  the  Navy  has 
not  submitted  the  shock  deficiency  correction  plan  nor 
demonstrated  the  effectiveness  of  corrective  actions  taken  to 
date. 

Assessment 

•  The  Navy  demonstrated  the  efficacy  of  LPD-17’s  collective 
protection  system,  its  countermeasure  wash-down  system, 
and  its  crew’s  ability  to  decontaminate  the  ship’s  equipment, 
personnel,  and  spaces,  and  to  conduct  amphibious  operations 
in  chemical  protective  suits  during  FOT&E. 

•  The  Navy  has  made  progress  in  improving  reliability  and 
availability  of  critical  ship  systems  affecting  communications 
and  propulsion  based  on  results  from  INSURV  and  limited 
reliability  testing  onboard  LPD-20.  However,  reliability 
and  availability  of  the  ship’s  critical  systems  require  further 
improvements  to  assure  the  ship  is  both  operationally  effective 
and  survivable. 

•  Since  IOT&E,  the  Navy  has  not  conducted  any  operational 
testing  to  demonstrate  improvements  to  LPD-17’s  capability 
to  defend  itself  against  the  threats  it  is  likely  to  encounter  to 
permit  a  reassessment  of  that  capability.  However,  operational 
testing  on  other  SSDS  Mk  2  platforms  has  revealed  similar 
combat  system  deficiencies  to  those  found  during  LPD-17’s 
IOT&E,  confirming  these  issues  are  not  LPD-17  specific. 
DOT&E’s  classified  report  to  Congress  in  November  2012 
titled  “Ship  Self-Defense  Operational  Mission  Capability 
Assessment  Report,”  provides  details.  The  Navy  is 
conducting  a  study  of  Probability  of  Raid  Annihilation  against 
anti-ship  cruise  missiles  and  expects  to  provide  their  report  in 
the  spring  of  2013. 

•  Although  improvements  have  been  made,  the  Navy 
has  not  yet  demonstrated  the  fully  effective  Command, 
Control,  Communications,  Computers,  and  Intelligence 
capabilities  needed  to  support  LPD-17  when  performing 


amphibious  assault  operations.  The  Joint  Staff  issued  the 
Joint  Interoperability  Certification  (with  conditions)  on 
October  26,  2012,  but  the  Navy  still  needs  to  successfully  test 
the  Advanced  Field  Artillery  Tactical  Data  System  onboard 
LPD-17. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
satisfied  recommendations  relating  to  interoperability 
testing  with  AV-8B  aircraft;  completion  of  chemical, 
biological,  and  radiological  defense  testing;  review  of  past 
INSURV  inspections;  initiation  of  the  Probability  of  Raid 
Annihilation  modeling  and  simulation  effort;  and  improving 
the  performance  of  the  SPS-48E  in  the  Advanced  Enclosed 
Mast  Structure.  The  Navy  should  act  on  the  remaining  10 
recommendations,  which  are  listed  below. 

FY08 

1.  Test  fixes  to  critical  systems  including  the  Shipboard  Wide 
Area  Network  and  review  the  effect  of  ship’s  manning, 
training,  and  logistics  support  on  the  reliability  and 
maintainability  of  ship  systems. 

FY09 

2.  Address  and  test  fixes  to  reliability  problems  with 
amphibious  support  equipment  and  propulsion  equipment 
during  FOT&E. 

3.  Continue  to  pursue  mitigations  to  address  integration 
problems  with  self  defense  in  multiple  warfare  areas. 

4.  Conduct  FOT&E  in  order  to  demonstrate  improvements  to 
performance  problems  related  to  the  Advanced  Enclosed 
Mast  Structure  (verify  installation  of  the  shroud  on  the 
SPS-48E  radar  corrects  performance  problems). 

FY10 

5.  Improve  reliability  of  critical  systems  including  gun 
systems,  Magnetic  Signature  Control  System,  and 
effectiveness  of  SSDS  Mk  2-based  combat  system. 

6.  Measure  Total  Ship  Operational  Availability  over  an 
extended  period  after  completing  reliability  improvements. 

FY11 

7.  Correct  remaining  deficiencies  from  Shock  Trial  Reports. 

8.  Complete  FOT&E  to  test  Information  Assurance. 

9.  Conduct  FOT&E  using  the  Advanced  Mine  Simulation 
System  to  determine  vulnerability  of  LPD-17  to  enemy 
mines. 

10.  Incorporate  outstanding  test  events  as  FOT&E  into  the 
LPD-17  Test  and  Evaluation  Master  Plan. 

•  FY12  Recommendations.  The  Navy  should: 

1.  Develop  an  FOT&E  test  plan  for  adequate,  rigorous  testing 
of  the  critical  ship  systems  that  must  perform  reliably  to 
assure  LPD-17  is  operationally  effective  and  survivable. 

2.  Complete  and  disseminate  the  Probability  of  Raid 
Annihilation  study. 
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Mark  XIIA  Identification  Friend  or  Foe  (IFF)  Mode  5 


Executive  Summary 

•  Independent  Mark  XIIA  Mode  5  Identification  Friend  or  Foe 
(IFF)  (referred  to  as  “Mode  5”)  programs  exist  in  each  U.S. 
Military  Service  as  well  as  some  NATO  countries.  Although 
not  a  joint  program,  the  Services  are  developing  equipment 
capable  of  employment  on  multiple  Service  platforms. 

-  Of  the  four  separate  Service  efforts,  the  Navy  has  the 
only  established  Acquisition  Category  II  program,  with 
incorporation  of  Service-specific  Mode  5  capability 
through  platform-specific  Engineering  Change  Proposals 
(ECPs). 

-  The  Army  and  Marine  Corps  will  leverage  the  Navy 
program  and  the  Air  Force  will  execute  individual  ECPs  on 
their  affected  hardware. 

•  Although  the  Services  are  designing  and  building  Mode  5 
systems  to  comply  with  NATO  and  DoD  IFF  standards, 
DOT&E  initiated  oversight  because  of  the  concern  that  the 
multiple  programs  and  vendors  add  risk  to  achieving  joint 
interoperability. 

•  The  Navy  conducted  an  IOT&E  of  Mode  5  capability  that 
included  significant  joint  Service  participation  in  FY12. 
However,  the  event  was  severely  truncated  due  to  adverse 
weather.  Lack  of  adequate  test  data  prevented  DOT&E 
from  fully  assessing  system  effectiveness  and  suitability 
under  realistic  operational  conditions.  However,  there  were 
sufficient  data  to  assess  the  performance  of  the  individual 
components  that  comprise  the  Navy  Mode  5  system.  Those 
components  met  their  performance  thresholds  and  the  Navy 
Acquisition  Executive  granted  full-rate  production  authority  to 
the  program  in  July  2012. 

•  Following  IOT&E,  the  Program  Office  released  new  software 
builds  for  both  its  transponder  and  interrogator  systems  to 
address  those  discrepancies  highlighted  during  the  operational 
test  and  subsequent  reporting.  Additional  testing  is  required 
to  assess  the  performance  of  these  software  fixes  as  well  as 
the  Mode  5  interoperating  with  both  existing  and  planned  IFF 
systems.  The  next  opportunity  to  conduct  that  testing  is  now 
planned  for  the  3QFY13.  Successful  planning  and  execution 
of  this  event  should  resolve  DOT&E  concerns  about  joint 
interoperability  and  identification  in  a  system-of-systems 
context. 

System 

•  The  Mark  XIIA  Mode  5  IFF  is  a  cooperative  identification 
system  that  uses  interrogators  and  transponders  located  on  host 
platforms  to  send,  receive,  and  process  friendly  identification 
data. 

•  Mode  5  is  a  military-only  identification  mode,  which  modifies 
the  existing  Mark  XII  Mode  4  IFF  (referred  to  as  “Mode  4”) 
system  and  addresses  known  shortcomings  of  the  legacy 
Mode  4  identification  mode.  Mode  5  will  eventually  replace 


1  -  Replacement  Card  for  Mode  5  Capability 

2  -  RT-1832  F/A-18  Interrogator 

3  -  APX-123  Navy  Transponder  (All  Platforms) 

4  -  UPX-41C  Shipboard  Interrogator 


Mode  4  and  allows  National  Security  Agency-certified  secure 
encryption  of  interrogations  and  replies.  Primary  system 
features  include: 

-  A  lethal  interrogation  format,  which  is  used  by  a  “shooter” 
prior  to  weapons  release  as  a  final  attempt  to  get  a  valid 
Mode  5  reply  from  the  target,  even  with  the  target’s 
interrogated  Mode  5  transponder  system  in  standby;  this  is 
intended  to  reduce  fratricide 

-  A  random-reply-delay,  which  prevents  overlapping  replies 
and  provides  better  display  discrimination  for  closely 
spaced  platforms 

•  Mode  5  offers  more  modern  signal  processing,  compatibility 
with  legacy  Mode  4  systems  and  civilian  air  traffic  control, 
and  secure  and  encrypted  data  exchange  through  the  use  of  the 
new  waveform. 

•  Mode  5  serves  as  a  component  of  the  combat  identification 
process  used  on  ground-based  systems  such  as  the 
Army’s  Patriot  missile  system,  sea-based  systems  such  as 
Aegis-equipped  ships,  and  military  aircraft  to  include  the  E-3 
Airborne  Warning  and  Control  System  and  E-2  Hawkeye 
command  and  control  platforms. 

•  Independent  Mode  5  programs  exist  in  each  U.S.  Military 
Service  as  well  as  some  NATO  countries.  Although  not  a  joint 
program,  the  Services  are  developing  equipment  capable  of 
employment  on  multiple  Service  platforms. 

-  Of  the  four  separate  Service  efforts,  the  Navy  has  the 
only  established  Acquisition  Category  II  program,  with 
incorporation  of  Service-specific  Mode  5  capability 
through  platform-specific  ECPs. 
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-  The  Army  and  Marine  Corps  will  leverage  the  Navy 
program,  and  the  Air  Force  will  execute  individual  ECPs 
on  its  affected  hardware. 

Mission 

The  Combatant  Commander  employs  the  Mode  5  system  to 
provide  positive,  secure,  line-of-sight  identification  of  friendly 
platforms  equipped  with  an  IFF  transponder.  In  the  future,  this 
system’s  information  will  be  combined  with  other  cooperative 


and  non-cooperative  combat  identification  techniques  in  order 
to  provide  identification  of  all  platforms  -  enemy,  neutral,  and 
friendly. 

Major  Contractor 

B  AE  Systems  -  Arlington,  Virginia 


Activity 

•  Although  the  Services  are  designing  and  building  Mode  5 
systems  to  comply  with  NATO  and  DoD  IFF  standards, 
DOT&E  initiated  oversight  because  of  the  concern  that  the 
multiple  programs  and  vendors  add  risk  to  achieving  joint 
interoperability. 

•  The  Navy  executed  IOT&E  on  their  Mode  5  system 
for  shipboard  interrogators  and  transponders  as  well  as 
aircraft  transponders  from  October  through  November 
2011.  The  Navy  executed  the  test  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  (TEMP) 
and  test  plan;  however,  poor  weather  and  technical  test 
execution  deficiencies  truncated  the  amount  of  test  data 
recorded.  There  were  sufficient  data  to  assess  the  performance 
of  the  Navy  Mode  5  system-under-test;  however  there  was 
insufficient  data  to  provide  a  full  assessment  of  the  Mode  5 
system-of-systems  interoperability  and  suitability. 

•  DOT&E  published  its  IOT&E  report  in  July  2012, 
assessing  the  Navy’s  Mode  5  hardware/software  as  well 
as  the  overarching  Mode  5  system-of-systems.  The  Navy 
Acquisition  Executive  approved  full-rate  production  of  the 
Navy  Mode  5  system  in  July  2012  following  the  Navy  Mode  5 
IOT&E. 

•  The  Army  and  Air  Force  are  developing  and  testing 
Service-specific  Mode  5  capabilities: 

-  The  Army  developed  a  Mode  5  Air  Defense  Interrogator 
for  the  Patriot  and  Sentinel  systems;  it  is  currently  in 
developmental  testing. 

-  The  Air  Force  is  developing  a  separate  Mode  5  interrogator 
for  the  E-3  Airborne  Warning  and  Control  System. 

-  The  Air  Force  Operational  Test  and  Evaluation  Center 
tested  the  integration  of  an  Air  Force-developed  Mode  5 
interrogator  and  transponder  on  the  F-15C/E  aircraft  in 
conjunction  with  the  Navy  IOT&E  in  November  2011. 

•  The  Navy  submitted,  and  DOT&E  approved,  a  revised  Test 
and  Evaluation  Master  Plan  that  will  ensure  that  Mode  5  is 
assessed  in  operationally  realistic  environments  that  include 
(in  addition  to  Navy  ship  and  aircraft  platforms)  a  variety  of 
Army,  Air  Force,  and  allied  systems  equipped  with  Mode  5 
capability. 

•  USD(AT&L)  and  DOT&E  worked  with  the  Services  to 
develop  and  approve  a  revised  Joint  Operational  Test 
Approach  (JOTA)  (version  2.0)  document  to  guide  Mode  5 
interoperability  testing  across  the  DoD. 
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-  Utilizing  the  approved  JOTA  guidance,  the  Navy  is 
currently  leading  development  of  a  test  concept  for  the 
conduct  of  an  operationally  realistic  test  of  Mode  5 
capability  in  2013. 

-  This  event  will  employ  a  variety  of  joint  Service  and  allied 
aircraft  equipped  with  interrogators  and  transponders  using 
representative  flight  profiles.  JOTA  efforts  are  critical  to 
informing  the  DoD-wide  FY14  Mode  5  Initial  Operational 
Capability  and  FY20  Full  Operational  Capability 
declarations. 

Assessment 

•  During  IOT&E,  Mode  5  demonstrated  significant 
improvement  over  the  existing  Mode  4.  However,  weather 
limited  the  scope  of  testing  and  prevented  a  complete 
operational  assessment.  Lack  of  adequate  test  data  prevented 
DOT&E  from  fully  assessing  system  effectiveness  and 
suitability  under  operationally  realistic  conditions.  However, 
there  were  sufficient  data  to  assess  the  performance  of  the 
individual  hardware  components  that  comprise  the  Navy 
Mode  5  system.  Those  components  met  their  individual 
performance  requirements  thresholds  during  the  IOT&E. 

•  Although  no  hardware  or  software  failures  occurred  during 
IOT&E,  DOT&E  observed  substantial  suitability  deficiencies, 
including  short  battery  life,  easily  triggered  anti-tamper 
features,  and  difficulty  loading  cryptographic  keys. 

•  The  IOT&E  highlighted  interoperability  concerns  between 
Mode  5  and  other  systems  onboard  Navy  ships.  These 
include: 

-  Shipboard  integration  problems  of  Navy  Mode  5 
equipment  into  the  larger  shipboard  Aegis  weapons  system, 
which  could  cause  incorrect  engagement  decisions  with 
potentially  severe  consequences. 

-  Problems  with  the  accurate  and  timely  flow  of 
Mode  5-derived  identification  information  between 
components  of  the  Navy  Cooperative  Engagement 
Capability  system. 

•  Following  IOT&E,  the  Program  Office  released  new  software 
builds  for  both  its  transponder  and  interrogator  systems  to 
address  those  discrepancies  highlighted  during  the  operational 
test  and  subsequent  reporting.  Additional  testing  is  required 
to  assess  the  performance  of  these  software  fixes  as  well  as 
the  Mode  5  interoperating  with  both  existing  and  planned  IFF 
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systems.  The  next  opportunity  to  conduct  that  testing  is  now 
planned  for  the  3QFY13.  Successful  planning  and  execution 
of  this  event  should  resolve  DOT&E  concerns  about  joint 
interoperability  and  identification  in  a  system-of-systems 
context. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
adequately  addressed  all  previous  recommendations. 

•  FY12  Recommendations. 

1 .  In  order  to  ensure  interoperability  between  interrogators, 
transponders,  and  combined  interrogator-transponders, 


the  Service  program  managers  must  continue  to  integrate 
their  test  schedules.  Additionally,  all  Services  must  fully 
participate  in  the  JOTA  evaluation  process  to  ensure  that 
Mode  5  capabilities  are  tested  in  a  realistic  joint  Service 
environment. 

2.  The  Navy  needs  to  address  problems  with  the  Mode  5 
shipboard  integration  on  the  Aegis  weapons  system  and 
the  information  flow  between  Mode  5  and  the  Cooperative 
Engagement  Capability  command  and  control  infrastructure 
to  ensure  that  the  Mode  5  system  capabilities  are  fully 
effective. 
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MH-60R  Multi-Mission  Helicopter 


Executive  Summary 

•  DOT&E  issued  a  combined  FOT&E  report  (MH-60R  and 
MH-60S)  in  April  2012  assessing  the  operational  effectiveness 
and  operational  suitability  of  selected  systems  of  the  MH-60R 
Pre-Planned  Product  Improvement  Program.  The  Ground 
Proximity  Warning  System  (GPWS)  and  the  Integrated 
Maintenance  Diagnostic  System  (IMDS)  are  operationally 
effective  and  operationally  suitable  for  all  MH-60R  missions. 
The  overall  assessment  of  the  MH-60R  airframe  for  all 
mission  areas  remains  operationally  effective,  operationally 
suitable,  and  survivable. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  commenced  two  tests  in  4QFY12.  Both  tests 
are  anticipated  to  complete  in  1QFY13: 

-  IOT&E  of  MF1-60R  equipped  with  the  Automatic  Radar 
Periscope  Detection  and  Discrimination  (ARPDD) 

Upgrade 

-  Testing  focused  on  previously  identified  Multi-spectral 
Targeting  System  (MTS)  deficiencies 

•  The  analyses  of  test  data  collected  during  IOT&E  of  MH-60R 
with  ARPDD  and  testing  of  MH-60R  with  MTS  are  still  in 
progress.  DOT&E  expects  to  issue  formal  test  reports  in 
2QFY13. 

System 

•  The  MH-60R  is  a  ship-based  helicopter  designed  to  operate 
from  cruisers,  destroyers,  frigates,  littoral  combat  ships,  and 
aircraft  carriers. 

•  It  incorporates  dipping  sonar  and  sonobuoy  acoustic 
sensors,  multi-mode  radar,  electronic  warfare  sensors,  a 
forward-looking  infrared  sensor  with  laser  designator,  and  an 
advanced  mission  data  processing  system. 


•  It  employs  torpedoes,  Hellfire  air-to-surface  missiles,  and 
crew-served  mounted  machine  guns. 

•  It  has  a  three-man  crew:  two  pilots  and  one  sensor  operator. 

Mission 

The  Maritime  Component  Commander  employs  the  MH-60R 

from  ships  or  shore  stations  to  accomplish  the  following: 

•  SUW,  Under  Sea  Warfare,  Area  Surveillance,  Combat 
Identification,  and  Naval  Surface  Fire  Support  missions 
previously  provided  by  two  different  helicopters  (SH-60B  and 
SH-60F) 

•  Support  missions  such  as  Search  and  Rescue  at-sea  and,  when 
outfitted  with  necessary  armament,  maritime  force  protection 
duties 

Major  Contractors 

•  Sikorsky  Aircraft  Corporation  -  Stratford,  Connecticut 

•  Lockheed  Martin  Mission  System  and  Sensors  -  Owego,  New 
York 


Activity 

•  DOT&E  issued  a  combined  FOT&E  report  (MH-60R 
and  MH-60S)  in  April  2012  assessing  the  operational 
effectiveness  and  operational  suitability  of  selected  systems 
of  the  MH-60R  Pre-Planned  Product  Improvement  Program. 
The  tested  systems  were  the  GPWS  and  the  IMDS.  COTF 
completed  testing  in  1QFY12.  COTF  conducted  the  testing  in 
accordance  with  the  DOT&E-approved  test  plan. 

•  COTF  commenced  IOT &E  of  MH-60R  with  the  ARPDD 
Upgrade  in  4QFY12.  COTF  conducted  the  testing  in 
accordance  with  a  DOT&E-approved  test  plan.  Testing  is 
anticipated  to  complete  in  1QFY13. 

•  COTF  commenced  testing  of  previously  identified  deficiencies 
of  the  MH-60R  with  MTS  in  4QFY12.  Testing  was  conducted 


in  accordance  with  a  DOT&E-approved  data  management  and 
analysis  plan  and  is  anticipated  to  complete  in  1QFY13. 

•  All  LFT&E  activities  have  been  completed  and  reported  in  the 
LFT&E  report  to  Congress  in  2008. 

Assessment 

•  The  GPWS  and  IMDS  are  assessed  to  be  operationally 
effective  and  operationally  suitable  for  all  MH-60R  missions. 
There  were  no  significant  operational  effectiveness  or 
operational  suitability  deficiencies  identified  during  testing. 
The  test  results  did  not  affect  any  prior  findings  on  the 
overall  operational  effectiveness,  operational  suitability,  or 
survivability  of  the  MH-60R  airframe  in  any  mission  area. 
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•  The  analysis  of  test  data  collected  during  IOT&E  of  MH-60R 
with  ARPDD  is  still  in  progress.  DOT&E  expects  to  issue  a 
formal  test  report  in  2QFY13. 

•  The  analysis  of  test  data  collected  during  testing  of  MH-60R 
with  MTS  is  still  in  progress.  The  scope  of  the  testing  was 
focused  on  previously  identified  MTS  deficiencies.  The  testing 
was  not  designed  to  assess  surface  warfare  mission  capability 
of  MH-60R  when  equipped  with  MTS  and  the  Hellfire  missile. 
DOT&E  expects  to  issue  a  formal  test  report  in  2QFY13. 


Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
addressed  the  four  previous  recommendations. 

•  FY12  Recommendation. 

1.  The  Navy  should  plan  to  conduct  FOT&E  to  assess 

MFl-60R’s  surface  warfare  capability  when  equipped  with 
MTS  and  the  Flellfire  missile. 
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MH-60S  Multi-Mission  Combat  Support  Helicopter 


Executive  Summary 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  completed  testing  of  selected  systems  of 
the  MF1-60S  Pre-Planned  Product  Improvement  (P3I) 

Program  in  1QFY12.  DOT&E  issued  a  combined  FOT&E 
report  (MF1-60R  and  MF1-60S)  in  April  2012  assessing  the 
operational  effectiveness  and  operational  suitability  of  these 
systems.  The  Active  Vibration  Control  System  (AVCS), 
Ground  Proximity  Warning  System  (GPWS),  and  Integrated 
Maintenance  Diagnostic  System  (IMDS)  are  operationally 
effective  and  operationally  suitable  for  all  MF1-60S  missions. 
The  overall  assessment  of  the  MF1-60S  airframe  remains 
operationally  effective,  operationally  suitable,  and  survivable 
for  all  mission  areas. 

•  COTF  completed  testing  of  the  MFI-60S  Block  2  Airborne 
Mine  Countermeasures  (AMCM)  System  in  4QF Y 1 1 . 

DOT&E  issued  an  operational  assessment  (OA)  report  in 
June  2012  assessing  operational  effectiveness  and  operational 
suitability  of  the  minehunting  capability  meant  to  be  provided 
by  the  AMCM  System  and  the  AN/AQS-20A  Sonar  Mine 
Detecting  Set  (AQS-20A). 

-  The  MFI-60S  helicopter  with  the  AQS-20A  sonar  is  not 
operationally  effective  or  suitable  because  the  helicopter  is 
underpowered  and  cannot  safely  tow  the  sonar  under  the 
wide  variety  of  conditions  necessary. 

-  As  observed  during  the  OA  and  developmental  testing,  the 
AQS-20A  does  not  meet  some  Navy  requirements. 

•  COTF  completed  two  test  events  in  4QFY12.  The  analysis  of 
test  data  collected  during  these  events  is  still  in  progress. 

-  Phase  A  (Shore-based  and  Training  Phase)  of  the  planned 
OA  of  the  MF1-60S  Block  2  AMCM  System  and  the 
Airborne  Laser  Mine  Detection  System  (ALMDS), 
commenced  in  2QFY12.  Preliminary  evaluation  of  data 
collected  during  the  OA  suggests  that  the  ALMDS  does  not 
meet  Navy  requirements  for  False  Classification  Density 
(FCD)  and  has  low  reliability.  DOT&E  expects  to  issue  a 
formal  test  report  in  1QFY13. 

-  The  Quick  Reaction  Assessment  (QRA)  of  MF1-60S  with 
the  20  mm  Gun  System  (Forward  Fixed  Firing  Weapon) 
commenced  in  3QFY12.  Preliminary  evaluation  of 
data  collected  during  the  QRA  suggests  that  the  20  mm 
Gun  System  may  provide  enhanced  Surface  Warfare 
performance  to  the  MFI-60S  helicopter.  DOT&E  expects 
to  issue  a  formal  test  report  in  1QFY13. 

•  COTF  commenced  testing  focused  on  previously  identified 
Multi-spectral  Targeting  System  (MTS)  deficiencies  in 
4QFY12;  testing  is  anticipated  to  complete  in  late  1QFY13. 
The  analysis  of  test  data  collected  during  testing  of  MF1-60S 
with  MTS  is  still  in  progress.  DOT&E  expects  to  issue  a 
formal  test  report  in  2QFY13. 


System 

•  The  MF1-60S  is  a  helicopter  modified  into  three  variants 
(Blocks)  from  the  Army  UFI-60L  Blackhawk.  It  is  optimized 
for  operation  in  the  shipboard/maritime  environment. 

•  The  Blocks  share  common  cockpit  avionics  and  flight 
instrumentation  with  the  MF1-60R. 

•  Installed  systems  differ  by  Block  based  on  mission: 

-  Block  1,  Fleet  Logistics  -  precision  navigation  and 
communications,  maximum  cargo  or  passenger  capacity 

-  Block  2  A/B,  AMCM  System  -  AMCM  system  operator 
workstation,  a  tether/towing  system,  and  the  two  MCM 
systems  currently  under  development;  ALMDS  for 
detection  and  classification  of  near-surface  mines  and 
the  Airborne  Mine  Neutralization  System  (AMNS)  for 
neutralization  of  in-volume  and  bottom  mines.  The 
AQS-20A  sonar  system  and  Organic  Airborne  and  Surface 
Influence  Sweep  are  no  longer  being  developed  for  use  in 
the  AMCM  System 

-  Block  3  A,  Armed  Helicopter  -  tactical  moving  map 
display,  forward-looking  infrared  with  laser  designator, 
crew-served  side  machine  guns,  dual-sided  Hellfire 
air-to-surface  missiles,  and  defensive  electronic 
countermeasures 

-  Block  3B,  Armed  Helicopter  -  Block  3A  with  addition  of 
tactical  datalink  (Link  16) 

•  P3I  components  add  Link  16  and  various  communication, 
navigation,  and  command  and  control  upgrades. 

Mission 

The  Maritime  Component  Commander  can  employ  variants  of 

MH-60S  from  ships  or  shore  stations  to  accomplish  the  following 

missions: 
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•  Block  1  -  Vertical  replenishment,  internal  cargo  and  personnel 
transport,  medical  evacuation,  Search  and  Rescue,  and  Aircraft 
Carrier  Plane  Guard 

•  Block  2  -  Detection,  classification,  identification  and/or 
neutralization  of  sea  mines  depending  on  which  AMCM 
systems  are  employed  on  the  aircraft 

•  Block  3  -  Combat  Search  and  Rescue,  Surface  Warfare, 
Aircraft  Carrier  Plane  Guard,  Maritime  Interdiction 
Operations,  and  Special  Warfare  Support 


Major  Contractors 

•  Sikorsky  Aircraft  Corporation  -  Stratford,  Connecticut 

•  Lockheed  Martin  Mission  System  and  Sensors  -  Owego,  New 
York 

•  Raytheon  Company,  Integrated  Defense  Systems  -  Tewksbury, 
Massachusetts 

•  Northrop  Grumman  Corporation  -  Melbourne,  Florida 


Activity 

•  COTF  completed  testing  of  the  AVCS,  GPWS,  and  IMDS 
improvements  in  1QFY12.  DOT&E  issued  a  combined 
FOT&E  report  (MH-60R  and  MH-60S)  in  April  2012 
assessing  the  operational  effectiveness  and  operational 
suitability  of  these  systems. 

•  COTF  completed  testing  of  the  MH-60S  Block  2A  AMCM 
System  and  the  AQS-20A  in  4QFY 1 1 .  DOT&E  issued  an  OA 
report  in  June  2012  assessing  operational  effectiveness  and 
operational  suitability  of  the  minehunting  capability. 

•  COTF  commenced  Phase  A  (Shore -based  and  Training  Phase) 
of  the  planned  OA  of  the  MH-60S  Block  2  AMCM  System 
and  the  ALMDS  in  2QFY12;  testing  completed  in  4QFY12. 
The  Navy  postponed  conduct  of  Phase  B  (Littoral  Combat 
Ship  (LCS)  Ship-based  Phase)  of  the  planned  OA  due  to  the 
unavailability  of  an  LCS  seaframe  to  facilitate  conduct  of 
MCM  mission  module  testing.  The  Navy  intends  to  conduct 
the  LCS  ship-based  phase  of  the  planned  ALMDS  and  AMNS 
OAs  in  conjunction  with  the  LCS  Technical  Evaluation 
scheduled  to  occur  in  FY14. 

•  COTF  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  test  plan. 

•  COTF  commenced  a  QRA  of  MH-60S  with  the  20  mm  Gun 
System  (Forward  Fixed  Firing  Weapon)  in  3QFY12;  testing 
completed  in  4QFY12.  COTF  conducted  the  assessment  in 
accordance  with  the  DOT&E-approved  data  management  and 
analysis  plan. 

•  COTF  commenced  testing  of  previously  identified  deficiencies 
of  the  MH-60S  with  MTS  in  4QFY12.  COTF  conducted 

the  testing  in  accordance  with  a  DOT&E-approved  data 
management  and  analysis  plan.  Testing  is  anticipated  to 
complete  in  1QFY13. 

•  All  LFT&E  activities  have  been  completed  and  reported  in  the 
LFT&E  report  to  Congress  in  2008. 

Assessment 

•  The  AVCS,  GPWS,  and  IMDS  are  assessed  to  be  operationally 
effective  and  operationally  suitable  for  all  MH-60S 
missions.  There  were  no  significant  operational  effectiveness 
deficiencies  identified  during  testing.  The  test  results  did 

not  affect  any  prior  findings  on  the  overall  operational 
effectiveness,  operational  suitability,  or  survivability  of  the 
MH-60S  airframe  in  any  mission  area. 


•  The  MH-60S  helicopter  and  AQS-20A  sonar  are  not 
operationally  effective  or  suitable  because  the  helicopter 
is  underpowered  and  cannot  safely  tow  the  sonar  under 
the  variety  of  conditions  necessary.  The  Chief  of  Naval 
Operations  recently  concluded  that  the  MH-60S  helicopter 
is  significantly  underpowered  for  the  safe  performance  of 
the  AMCM  tow  mission  and  provides  limited  tactical  utility 
relative  to  the  risk  to  aircrew,  and  cancelled  that  MH-60S 
mission.  The  decision  to  cancel  the  AMCM  tow  mission 
affects  employment  of  both  the  AQS-20A  sonar  and  Organic 
Airborne  and  Surface  Influence  Sweep. 

•  As  observed  during  the  OA  and  developmental  testing,  the 
AQS-20A  does  not  meet  all  Navy  requirements  in  all  operating 
modes.  Contact  depth  (vertical  localization)  errors  exceeded 
Navy  limits  in  all  AQS-20A  operating  modes.  FCD  also 
exceeded  Navy  limits  in  two  of  three  search  modes. 

•  The  analysis  of  test  data  collected  during  Phase  A  of  the  OA 
of  the  MH-60S  and  ALMDS  is  still  in  progress.  Preliminary 
evaluation  of  data  collected  during  the  OA  suggests  that 

the  ALMDS  does  not  meet  Navy  requirements  for  FCD  or 
reliability.  DOT&E  expects  to  issue  a  formal  test  report 
in  2QFY13.  Phase  B  testing  was  originally  intended  to 
provide  early  operational  testing  insight  into  the  operational 
effectiveness  and  suitability  of  AMCM  systems  when 
operating  from  an  LCS,  and  to  identify  risk  to  the  successful 
completion  of  IOT&E.  However,  the  Navy’s  cancellation  of 
Phase  B  testing  will  eliminate  these  intended  benefits. 

•  The  analysis  of  test  data  collected  during  the  QRA  of  MH-60S 
with  the  20  mm  Gun  System  (Forward  Fixed  Firing  Weapon) 
is  still  in  progress.  Preliminary  evaluation  of  data  collected 
during  the  QRA  suggests  that  the  20  mm  Gun  System  may 
provide  enhanced  Surface  Warfare  performance  to  the 
MH-60S  helicopter.  DOT&E  expects  to  issue  a  formal  test 
report  in  1QFY13. 

•  The  analysis  of  test  data  collected  during  testing  of  MH-60S 
with  MTS  is  still  in  progress.  The  scope  of  the  testing  was 
focused  on  previously  identified  MTS  deficiencies.  The  testing 
was  not  designed  to  assess  Surface  Warfare  mission  capability 
of  MH-60S  when  equipped  with  MTS  and  the  Hellfire  missile. 
DOT&E  expects  to  issue  a  formal  test  report  in  2QFY13. 


178  MH-60S 


NAVY  PROGRAMS 


Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
addressed  8  of  the  1 1  previous  recommendations.  The  Navy 
should  still: 

1.  Investigate  solutions  and  correct  AN/AQS-20A  FCD  and 
Vertical  Localization  deficiencies  prior  to  IOT&E. 

2.  Investigate  solutions  and  correct  the  ALMDS  FCD 
deficiency  prior  to  IOT&E. 

3.  Conduct  LCS  ship-based  phases  of  the  planned  OA  of  the 
MH-60S  Block  2  and  ALMDS  as  well  as  an  OA  of  the 


MH-60S  Block  2  and  AMNS  MCM  systems  in  FY13  to 
reduce  risk  to  the  LCS  MCM  Mission  Package  IOT&E. 

•  FY12  Recommendation. 

1.  The  Navy  should  plan  to  conduct  FOT&E  to  assess 
MH-60S’s  surface  warfare  capability  when  equipped  with 
MTS  and  the  Hellfire  missile. 
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Mk  48  Advanced  Capability  (ADCAP) 
Torpedo  Modifications 


Executive  Summary 

•  In  FY12,  the  Navy  continued  operational  testing  of  the 
Advanced  Processor  Build  (APB)  Spiral  4  operational 
software  for  the  Mk  48  Advanced  Capability  (ADCAP) 
Modification  (Mod)  7  Common  Broadband  Advanced 
Sonar  System  (CBASS)  torpedo  and  Mk  48  ADCAP  Mod  6 
Advanced  Common  Torpedo  (ACOT).  OT&E,  which  began 
in  FY11,  is  expected  to  continue  into  early  FY13. 

•  Since  the  commencement  of  operational  testing  in  F  Y 1 1 , 
the  Navy  has  fired  approximately  330  torpedoes  equipped 
with  Spiral  4  tactical  software  for  training  and  testing  at 
both  surface  and  submerged  targets  in  a  variety  of  different 
environmental  and  tactical  scenarios.  These  scenarios  have 
included  targets  deploying  multiple  static  countermeasures, 
targets  deploying  the  mobile  countermeasure,  and  targets 
designed  to  emulate  the  threat  identified  in  the  Navy’s  Urgent 
Operational  Needs  Statement  (UONS)  of  March  19,  2010. 

•  Because  the  Navy  did  not  complete  developmental  testing 
before  early  fielding  and  before  commencing  operational 
testing  of  the  Mk  48  Spiral  4  torpedo,  operational  testers, 
the  Navy’s  laboratories,  and  fleet  users  identified  several 
performance  deficiencies  during  operational  testing.  The 
Navy  interrupted  the  operational  testing  and  issued  new  Mk  48 
Spiral  4  torpedo  operational  software  with  the  intention  of 
fixing  identified  deficiencies. 

System 

•  The  Mk  48  ADCAP  torpedo  is  the  only  Anti-Submarine 
Warfare  and  Anti-Surface  Ship  Warfare  weapon  used  by  U.S. 
submarines.  Mk  48  ADCAP  torpedo  modifications  are  a  series 
of  hardware  and  software  upgrades  to  the  weapon. 

•  Mk  48  Mod  6,  Mod  6  Spiral  1,  Mod  6  ACOT  -  Guidance  and 
Control  Box,  and  Mod  7  CBASS  Phase  I  are  fielded  torpedoes. 

•  Mk  48  Mod  7  CBASS  upgrades  the  Mk  48  ACOT  with  a 
new  sonar  designed  to  improve  torpedo  effectiveness  through 
future  software  upgrades.  Phase  1  torpedoes  deliver  the  initial 
hardware  and  software;  Phase  2  torpedoes  are  required  to 
deliver  full  capability. 


•  The  software  developed  for  CBASS  Phase  2  is  designated 
APB  Spiral  4.  The  Navy  subsequently  determined  that 
Spiral  4  software  can  run  on  ACOT  weapons  as  well.  As  a 
result,  the  Navy  is  testing  Spiral  4  on  both  CBASS  and  ACOT 
weapons.  The  Navy  has  authorized  the  limited  fielding  of 
Mk  48  Spiral  4  torpedoes. 

•  CBASS  is  a  co-development  program  with  the  Royal 
Australian  Navy. 

Mission 

The  Submarine  Force  employs  the  Mk  48  ADCAP  torpedo  as  a 

long-range,  heavy-weight  weapon: 

•  For  destroying  surface  ships  or  submarines 

•  In  both  deep-water  open  ocean  and  shallow-water  littoral 
environments 

Major  Contractor 

Lockheed  Martin  Sippican  Inc.  -  Marion,  Massachusetts 


Activity 

•  In  FY12,  the  Navy  continued  operational  testing  of  the  APB 
Spiral  4  operational  software  for  the  Mk  48  ADCAP  Mod  7 
(CBASS)  torpedo  and  Mk  48  ADCAP  Mod  6  (ACOT). 
OT&E,  which  began  in  FY11,  is  expected  to  continue  into 
early  FY13. 

•  Since  the  commencement  of  operational  testing  in  FY 1 1 , 
the  Navy  has  fired  approximately  330  torpedoes  equipped 


with  Spiral  4  tactical  software  for  training  and  testing  at 
both  surface  and  submerged  targets  in  a  variety  of  different 
environmental  and  tactical  scenarios.  These  scenarios  have 
included  targets  deploying  multiple  static  countermeasures, 
targets  deploying  the  mobile  countermeasure,  and  targets 
designed  to  emulate  the  threat  identified  in  the  Navy’s  UONS 
of  March  19,  2010. 
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•  Relevant  prior  activity  includes: 

-  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  observed  and  analyzed  the  results  of  the 
in-water  Mk  48  Spiral  4  exercises  and  developmental 
testing  from  January  to  February  2011.  COTF  also 
conducted  modeling  and  simulation  assessments  using  the 
Weapons  Analysis  Facility  (WAF)  located  at  the  Naval 
Undersea  Warfare  Center,  Newport,  Rhode  Island,  to 
examine  Mk  48  Spiral  4  performance  in  baseline  warfare 
scenarios. 

-  The  Navy  released  the  Mk  48  Spiral  4  torpedo  for  limited 
operational  use  in  March  2011. 

-  DOT&E  delivered  an  Early  Fielding  report  to 
Congressional  defense  committees  in  March  2011. 

-  The  Navy  has  shifted  to  a  Technical  Insertion  (TI)  and 
APB  model  for  torpedo  development.  The  Navy  intends 
the  hardware  modernizations  (TIs)  to  address  component 
obsolescence  and  to  enable  future  capability.  The  Navy 
intends  the  new  operational  software  (APBs)  to  use  the 
new  Tl  hardware  and  to  improve  the  torpedo’s  capability. 

•  The  Navy  updated  the  Joint  Test  and  Evaluation  Master  Plan 
to  cover  the  Spiral  4  with  Mk  48  ADCAP  CBASS  and  Mk  48 
ADCAP  ACOT,  and  to  address  the  UONS  threat.  DOT&E 
approved  the  updated  Test  and  Evaluation  Master  Plan  on 
January  13,  2012. 

•  The  Navy  completed  development  of  a  Submarine-Launched 
Countermeasure  Emulator  (SLACE)  to  support  Mk  48 
Spiral  4  testing.  The  SLACE  emulator  enables  the  Navy 

to  conduct  more  realistic  torpedo  operational  testing 
against  threat  submarine  surrogates  that  can  employ  mobile 
countermeasures.  The  Navy  also  developed  a  Steel  Diesel 
Electric  Submarine  surrogate  to  evaluate  torpedo  performance 
against  submarine  threats  in  limited  operational  scenarios. 

•  DOT&E  approved  the  OT&E  initial  test  plan  for  Mk  48 
Spiral  4  on  July  14,  2011,  to  support  the  initial  operational  test 
events.  Because  the  Navy  was  unable  to  identify  future  test 
locations  and  test  resources,  and  provide  the  execution  details 
of  the  operational  scenarios,  DOT&E  required  the  Navy  to 
update  the  test  plan  once  the  follow-on  testing  was  planned 
and  before  conducting  the  remainder  of  the  operational  testing. 
DOT&E  approved  updated  test  plans  on  June  15,  2012,  and 
August  24,  2012.  DOT&E  also  required  the  Navy  to  submit  a 
final  update  to  the  test  plan  once  the  details  of  the  remainder  of 
the  Mk  48  Spiral  4  testing  were  known. 

•  The  Navy  conducted  the  first  phase  of  Spiral  4  operational 
testing  in  conjunction  with  FOT&E  on  the  Virginia  class 
submarine  in  March  2011  off  Maui  and  at  the  Pacific 
Missile  Range  Facility  off  Kauai,  Hawaii.  Submarines  fired 
17  Spiral  4  weapons  with  software  versions  3x.4.3  and  3x.4.4. 

•  In  June  2011,  the  Navy  conducted  10  firings,  with  Spiral  4 
software  version  3x.4.4,  off  southern  California. 

•  In  September  2011,  the  Navy  conducted  10  additional  Mk  48 
Spiral  4  torpedo  developmental  test  events  using  the  Steel 
Diesel  Electric  Submarine  target  surrogate  at  a  shallow-water 
site  off  the  Virginia  coast.  The  purpose  was  to  gain  additional 
torpedo  performance  information  against  the  UONS  threat. 


•  In  December  2011,  the  Navy  issued  Mk  48  Spiral  4  torpedo 
software  version  3x.4.6  to  correct  problems  identified  in  the 
completed  testing  and  by  fleet  operators.  In  order  to  avoid 
the  costly  repetition  of  all  completed  operational  testing, 
the  Navy’s  testers  evaluated  the  effects  of  these  changes  on 
the  torpedo’s  performance.  Test  events  where  performance 
would  likely  be  affected  by  the  new  software  change  were 
invalidated  from  the  operational  test  database  and  retesting 
was  incorporated  into  future  test  periods. 

•  The  Navy  conducted  13  firings  with  the  new  Spiral  4  software, 
version  3x.4.6,  in  June  2012  in  the  Narragansett  Bay  Operating 
Area.  This  test  was  held  in  conjunction  with  a  Tactical 
Development  Exercise  that  featured  another  12  torpedo  firings. 

•  In  September  2012,  the  Navy  conducted  11  Spiral  4  shots 
with  software  version  3x.4.6  at  two  sites  off  Maui,  Hawaii. 

An  Australian  Collins  class  diesel  submarine  served  as 
the  target  and  four  of  the  runs  featured  the  SLACE  mobile 
countermeasure  emulator. 

•  During  FY12,  the  Navy  employed  Spiral  4  weapons  during 
four  Submarine  Command  Course  exercises  at  the  Atlantic 
Undersea  Test  and  Evaluation  Center  and  the  Pacific  Missile 
Range  Facility.  To  conserve  test  resources,  DOT&E  agreed 
to  utilize  these  torpedo  events  as  regression  testing  to  evaluate 
the  performance  of  the  Mk  48  Spiral  4  in  some  deep-water 
scenarios. 

•  In  October  2012,  the  Navy  conducted  an  additional  three 
Spiral  4  shots  with  software  version  3x.4.6  off  Cape  Cod, 
Massachusetts. 

•  The  Navy  accredited  the  WAF,  located  at  the  Navy  Undersea 
Warfare  Center,  in  August  2012  to  compare  the  performance 
of  the  two  hardware  versions  of  the  Mk  48  torpedo  that  use  the 
Spiral  4  software. 

•  The  Navy  conducted  two  Mk  48  Sink  Exercise  (SINKEX) 
events  in  FY12,  using  war-shot  torpedoes.  Both  SINKEXs 
were  executed  by  allied  submarine  forces.  A  Canadian 
submarine  fired  an  Mk  48  Mod  4M  (Mk  48  version  sold  to 
allies),  while  an  Australian  submarine  fired  a  Mk  48  Mod  7. 
These  test  events  confirmed  the  warhead  performance  of 
in-service  and  stored  Mk  48  torpedoes. 

Assessment 

•  The  Navy’s  Quick  Reaction  Assessment  and  WAF  testing 
of  the  Mk  48  Spiral  4  torpedo  enabled  a  limited  assessment 
of  its  performance.  DOT&E’s  report  on  the  early  fielding 
assessed  that  testing  indicated  the  Mk  48  Spiral  4  has  a 
limited  capability,  under  certain  operational  conditions, 
against  the  threat  identified  in  the  UONS;  however,  the  Navy 
did  not  have  adequate  threat  surrogates  for  the  evaluation. 
DOT&E’s  assessment  also  reported  that  the  Spiral  4  torpedo 
did  not  demonstrate  expected  improvements  over  the  legacy 
torpedo,  and  may  degrade  current  capability  in  certain  warfare 
scenarios. 

•  Additional  information  on  Mk  48  Spiral  4  performance  can 
be  found  in  DOT&E’s  classified  Mk  48  ACOT  and  CBASS 
Spiral  4  Early  Fielding  Report  dated  March  18,  2011. 
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•  During  operational  testing,  the  Navy’s  testers  and  laboratories 
discovered  several  torpedo  deficiencies  not  identified  during 
developmental  testing  that  resulted  in  the  Navy  developing 
software  corrections  intended  to  fix  the  identified  deficiencies. 
The  Navy  used  the  WAF  for  assessing  the  software  changes, 
issued  a  new  software  version,  and  continued  with  the 
operational  testing.  As  the  WAF  consistently  overestimates 
performance  and  was  not  accredited  for  evaluating  torpedo 
effectiveness  or  for  operational  testing  of  the  Mk  48  Spiral  4, 
DOT&E  assesses  this  development  approach  as  high  risk 

for  adequately  predicting  satisfactory  torpedo  in-water 
performance. 

•  The  Navy  and  DOT&E  are  assessing  the  completed  Mk  48 
Spiral  4  test  events  for  operational  realism  and  validity 
incrementally  as  the  fleet  training  and  test  events  are 
completed.  Most  fleet  training  events  have  been  too  structured 
or  lacked  the  necessary  post  firing  operational  conditions 

to  meet  required  torpedo  test  conditions.  Navy  testers  are 
working  with  fleet  trainers  to  improve  the  post  torpedo  firing 
operational  realism. 

•  Due  to  delays  in  completing  the  development  of  the  SLACE 
mobile  countermeasure  surrogate  and  the  Navy’s  focus 

on  the  UONS  threat,  the  Navy  did  not  conduct  adequate 
developmental  testing  against  SLACE-like  countermeasures. 
DOT&E  assessed  that  Mk  48  Spiral  4  performance  against 
SLACE-like  threats  is  high  risk  because  the  Program  Office 
completed  little  in-water  developmental  testing.  Assessment 
of  the  Mk  48  Spiral  4  operational  testing  with  SLACE, 
conducted  in  September  2012,  is  in  progress. 

•  Due  to  the  shortage  of  available  test  submarine  shooters  and 
targets,  the  Navy  continues  to  have  difficulty  scheduling  and 
planning  adequate  torpedo  operational  test  events.  Thus, 

Navy  testers  have  been  unable  to  provide  the  execution  details 
for  completing  operationally  realistic  events  for  all  required 
Mk  48  Spiral  4  test  events.  As  a  result,  DOT&E  has  required 


the  Navy  to  submit  updates  to  the  test  plan  once  the  event 
details  are  known  and  approved  the  testing  event  by  event. 

•  Initial  regression  and  operational  testing  results  indicate  Mk  48 
performance  in  deep-water  and  shallow-water  areas  has  not 
substantially  changed;  however,  insufficient  testing  has  been 
completed  to  allow  a  statistically  significant  assessment. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
addressed  six  of  the  nine  previous  Annual  Report 
recommendations.  The  three  outstanding  recommendations 
are  as  follows: 

1 .  While  some  improvements  have  been  made  by  conducting 
regression  testing  in  conjunction  with  scheduled  fleet 
training  events  and  by  using  WAF  simulations,  the  Navy 
should  continue  to  address  reducing  in-water  test  delays  and 
improve  the  WAF  simulations. 

2.  As  the  Navy  continues  to  conduct  only  limited  torpedo 
training  and  testing  in  shallow  waters,  they  should  develop 
shallow-water  test  and  training  areas  and  modernize  the 
exercise  torpedo  locating  and  recovery  systems. 

3.  The  Navy  should  complete  development  of  threat 
representative  target  and  countermeasure  surrogates  for 
torpedo  testing.  In  addition  to  representing  the  physical  and 
signature  characteristics  of  the  threat,  the  surrogate  should 
be  capable  of  emulating  appropriate  operational  profiles  of 
the  threat. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Plan  and  conduct  adequate  developmental  testing  before 
starting  operational  testing. 

2.  Continue  conducting  the  Mk  48  Spiral  4  torpedo  testing 
in  FY13.  Testing  should  include  the  evaluation  of  torpedo 
performance  against  submarine  surrogates  for  the  small 
diesel-electric  threat. 
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Mk  54  Lightweight  Torpedo 


Executive  Summary 

•  In  August  to  September  2011,  the  Navy  fired  22  Mk  54  Block 
Upgrade  (BUG)  torpedoes  against  a  Steel  Diesel  Electric 
Submarine  surrogate  target  and  against  U.S.  attack  submarine 
targets  in  order  to  address  the  March  2010  Navy  Fifth  Fleet 
Urgent  Operational  Need  Statement  (UONS).  Based  on 

the  results  of  this  test,  the  Navy  revised  the  Mk  54  BUG 
tactical  software,  conducted  an  additional  phase  of  in-water 
developmental  testing  in  November  2011,  and  completed  a 
limited  release  of  the  weapon  to  the  fleet. 

•  DOT&E  issued  an  Early  Fielding  report  on  January  12,  2012. 
DOT&E  reported  that  based  on  completed  testing,  crews 
employing  the  Mk  54  have  a  limited  capability  against  the 
UONS  threat  under  favorable  targeting  and  environmental 
conditions.  DOT&E  also  reported  that  the  Navy’s  testing  was 
completed  under  best-case  scenarios,  and  the  Navy  did  not 
have  an  adequate  threat  surrogate  for  the  UONS  threat.  For 
additional  details,  see  DOT&E’s  classified  report. 

•  The  Navy  did  not  complete  adequate  in-water  or  modeling 
and  simulation  developmental  testing  of  the  Mk  54  BUG 
as  planned.  As  the  Program  Office  shifted  resources  to 
demonstrate  that  the  Mk  54  BUG  has  a  capability  against 
the  UONS  emerging  submarine  threat,  testing  focused  on 
the  UONS  threat  scenarios  vice  the  operational  scenarios  for 
which  the  Mk  54  BUG  was  originally  intended. 

•  The  Navy  began  operational  testing  on  the  Mk  54  with  BUG 
software  in  March  2012. 

System 

•  The  Mk  54  Lightweight  Torpedo  is  the  primary 
Anti-Submarine  Warfare  weapon  used  by  U.S.  surface  ships, 
fixed-wing  aircraft,  and  helicopters. 

•  The  Mk  54  combines  the  advanced  sonar  transceiver  of  the 
Mk  50  torpedo  with  the  legacy  warhead  and  propulsion  system 
of  the  older  Mk  46.  The  Mk  46  and  Mk  50  torpedoes  can  be 
converted  to  an  Mk  54  via  an  upgrade  kit. 

•  The  Mk  54  sonar  processing  is  an  expandable, 
open-architecture  system.  It  combines  algorithms  from 
the  Mk  50  and  Mk  48  torpedo  programs  with  the  latest 
commercial  off-the-shelf  technology. 

•  The  Navy  designed  the  Mk  54  sonar  processing  to  operate 
in  shallow-water  environments  and  in  the  presence  of  sonar 
countermeasures. 


•  The  Navy  has  designated  the  Mk  54  torpedo  to  replace 
the  Mk  46  torpedo  as  the  payload  section  for  the  Vertical 
Launched  Anti-Submarine  Rocket  for  rapid  employment  by 
surface  ships. 

•  The  High- Altitude  Anti-Submarine  Warfare  Weapons 
Capability  program  will  provide  an  adapter  kit  to  permit 
long-range,  high-altitude,  GPS-guided  deployment  of  the 
Mk  54  by  a  P-8A  Maritime  Patrol  Aircraft. 

•  The  Mk  54  BUG  is  a  software  upgrade  to  the  Mk  54  baseline 
torpedo  designed  to  correct  deficiencies  identified  during  the 
2004  Mk  54  IOT&E. 

•  The  Navy  is  planning  a  series  of  near-term  improvements 
to  the  Mk  54,  including  an  improved  sonar  array  and  block 
upgrades  to  the  tactical  software. 

Mission 

Navy  surface  ships  and  aircraft  employ  the  Mk  54  torpedo  as 

their  primary  anti-submarine  weapon: 

•  For  offensive  purposes,  when  deployed  by  Anti-Submarine 
Warfare  aircraft  and  helicopters 

•  For  defensive  purposes,  when  deployed  by  surface  ships 

•  In  both  deep-water  open  ocean  and  shallow-water  littoral 
environments 

•  Against  fast,  deep-diving  nuclear  submarines;  and  slow 
moving,  quiet,  diesel-electric  submarines 

Major  Contractor 

Raytheon  Integrated  Defense  Systems  -  Tewksbury, 

Massachusetts 


Activity 

•  The  Navy  started  operational  testing  of  the  Mk  54  BUG  the  Navy  early  fielded  in  January  2012  to  address  the  Fifth 

torpedo  in  FY12.  The  operational  testing  is  being  conducted  Fleet  UONS  threat.  Relevant  prior  activity  includes: 

with  the  same  version  of  the  torpedo’s  tactical  software  that 
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-  The  Navy  developed  a  Steel  Diesel  Electric 
Submarine  surrogate  to  evaluate  torpedo  performance 
against  stationary  submarine  threats  in  limited 
operational  scenarios.  The  Navy  also  developed  a 
Submarine-Launched  Countermeasure  Emulator  to 
support  torpedo  testing.  The  emulator  enables  the  Navy 
to  conduct  realistic  torpedo  operational  testing  against 
threat  submarine  surrogates  that  can  employ  mobile 
countermeasures . 

-  In  August  to  September  2011,  the  fleet  fired  22  Mk  54 
BUG  torpedoes  with  software  version  42.B.1  against 
a  Steel  Diesel  Electric  Submarine  surrogate  target  and 
against  U.S.  attack  submarine  targets.  Based  on  the  results 
of  this  test,  the  Navy  issued  Mk  54  BUG  software  version 
42. B. 2  to  correct  some  identified  performance  problems, 
conducted  an  additional  phase  of  in-water  testing  in 
November  2011,  and  fielded  the  Mk  54  early  for  limited 
use  in  January  2012. 

•  DOT&E  issued  a  classified  Early  Fielding  report  on 
January  12,  2012.  DOT&E  reported  that  based  on  completed 
testing,  crews  employing  the  Mk  54  have  a  limited  capability 
against  the  UONS  threat  under  favorable  targeting  and 
environmental  conditions.  DOT&E  also  reported  that  the 
Navy’s  testing  was  completed  under  best-case  scenarios,  and 
the  Navy  did  not  have  an  adequate  threat  surrogate  for  the 
UONS  threat.  For  additional  details,  see  DOT&E’s  classified 
report. 

•  During  FY 1 1  and  FY12,  the  Navy  updated  the  Mk  54  BUG 
Test  and  Evaluation  Master  Plan  (TEMP)  to  address  both  the 
new  testing  required  for  the  UONS  threat  and  the  planned 
correction  of  major  deficiencies  identified  during  the  2004 
IOT&E.  DOT&E  approved  the  Mk  54  BUG  TEMP  in 
December  2012. 

•  DOT&E  approved  the  Navy’s  Operational  Test  Plan  on 
February  24,  2012,  to  cover  the  first  phase  of  operational 
testing.  Because  Navy  testers  could  not  identify  the  test 
execution  details  of  all  planned  future  test  events,  DOT&E 
approved  the  test  plan  for  the  first  event  and  required  it  to 
be  updated  when  the  execution  details  could  be  defined  for 
the  future  test  events.  DOT&E  approved  an  update  to  the 
test  events  on  July  31,  2012,  and  expects  a  final  test  plan 
update  to  cover  the  remainder  of  the  operational  testing  in 
early  F  Y 1 3 .  The  Navy  conducted  the  first  phase  of  BUG 
operational  testing,  designated  OT-B1A,  off  the  coast  of 
southern  California  in  March  2012.  Three  weapons  were  fired 
by  an  Arleigh  Burke  class  destroyer  and  five  were  dropped  by 
MH-60R  helicopters.  Another  five  weapons  were  intended  to 
be  dropped  by  P-3C  aircraft,  but  those  events  were  cancelled 
due  to  aircraft  material  problems.  After  the  testing,  the  Navy 
declared  the  MH-60R  runs  invalid  due  to  testing  irregularities. 

•  The  Navy  conducted  the  second  phase  of  BUG  operational 
testing  off  Cape  Cod,  Massachusetts,  in  August  2012.  The 
P-8A  aircraft  delivered  eight  weapons;  MH-60R  helicopters 
dropped  another  six  weapons.  Three  more  planned  torpedo 
runs  were  not  completed. 


•  The  Navy  conducted  the  third  scheduled  phase  of  BUG 
operational  testing  off  Maui,  Hawaii,  in  September  2012. 

P-8  A  aircraft  delivered  eight  weapons  and  SH-60B  helicopters 
dropped  four  weapons.  An  additional  two  runs  were  not 
completed. 

•  The  Navy  is  planning  an  additional  test  event  to  complete  the 
remaining  Mk  54  BUG  testing  in  3QFY13. 

•  Asa  result  of  concerns  about  warhead  performance  and 
changes  to  the  warhead  exploder,  DOT&E  placed  the  Mk  54 
on  live  fire  oversight  in  2010.  The  Navy  had  completed  the 
Mk  54  BUG  exploder  modification  and  testing;  therefore, 
DOT&E  agreed  to  the  Navy’s  proposal  to  develop  a  LFT&E 
plan  starting  with  the  Mk  54  Mod  1  version  of  the  torpedo. 

The  Mk  54  Program  Office  met  with  DOT&E  in  July  2012, 
and  held  an  LFT&E  meeting  in  August  2012  to  develop  an 
adequate  lethality  program  for  the  Mk  54  Mod  1  torpedo. 
DOT&E  is  working  with  the  Navy  to  establish  a  strategy  for 
LFT&E  to  support  the  FY13  Mk  54  Milestone  B. 

•  The  Navy  plans  to  continue  the  Mk  54  program  with  the 
Mk  54  Mod  1  torpedo  and  plans  to  approve  a  new  set  of 
requirements  documents  in  FY13. 

•  In  September  2012,  the  Navy  conducted  the  first  Mk  54 
Service  Weapons  Test  in  an  attempt  to  assess  the  performance 
of  the  warhead.  The  result  of  the  event  is  under  evaluation. 

Assessment 

•  The  Navy  originally  planned  the  Mk  54  BUG  software  to 
improve  Mk  54  classifier  and  tracker  performance  and  to 
resolve  IOT&E  Mk  54  deficiencies.  The  UONS  emerging 
threat  provided  the  incentive  for  the  Navy  to  accelerate  the 
development  and  fielding  of  the  Mk  54  BUG  software. 

•  The  operational  profile  of  the  UONS  emerging  threat  and 
the  resulting  changes  to  the  torpedo’s  final  homing  software 
and  exploder  requires  further  testing  to  confirm  Mk  54 
performance,  to  include  additional  target  operational  scenarios, 
additional  submarine  target  types,  and  the  assessment  of 

the  torpedo’s  final  terminal  homing  and  impact  of  the  target 
(set-to-hit). 

•  Since  safety  concerns  prevent  using  manned  submarines  for 
set-to-hit  testing,  the  Navy  developed  an  unmanned  Steel 
Diesel  Electric  Submarine  target.  The  Navy  is  using  this 
surrogate  for  both  set-to-hit  and  set-not-to-hit  testing.  The 
Steel  Diesel  Electric  Submarine  target  has  different  signature 
characteristics  than  the  UONS  emerging  threat,  thus  this 
surrogate  is  of  limited  utility  in  assessing  torpedo  operational 
performance  for  the  UONS.  However,  completing  set-to-hit 
terminal  homing  testing  may  address  some  unresolved  test 
scenarios  identified  in  the  IOT&E.  Mk  54  BUG  performance 
in  these  previously  unresolved  test  areas  will  affect  the  overall 
effectiveness  and  suitability  of  the  torpedo  against  other 
submarine  threats. 

•  The  Navy  did  not  complete  adequate  in-water  developmental 
testing  of  the  Mk  54  BUG.  As  the  Program  Office  shifted 
resources  to  demonstrate  that  the  Mk  54  BUG  has  a  capability 
against  the  UONS  emerging  submarine  threat,  testing  focused 
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on  the  UONS  threat  scenarios  vice  the  operational  scenarios 
for  which  the  Mk  54  BUG  was  originally  intended. 

•  To  date,  the  Navy’s  emerging  threat  test  scenario  execution 
was  structured  so  that  attacking  crews  had  near  perfect 
knowledge  of  the  target’s  location.  In  addition,  the  Navy 
conducted  UONS  testing  in  a  relatively  benign  area  that 
minimized  torpedo  interactions  with  the  bottom  or  false 
contacts.  Testing  in  these  structured  scenarios  indicates  the 
Mk  54  BUG  likely  has  a  limited  capability  against  the  Steel 
Diesel  Electric  Submarine  surrogate  target.  The  Mk  54  BUG 
performance  in  other  environmental  areas  and  against  some 
operationally  realistic  target  scenarios  is  being  tested  in 
FY12/13. 

•  The  Navy  is  using  a  1995  Operational  Requirements 
Document,  supplemented  with  sponsor  clarification  letters,  as 
the  reference  to  develop  improvements  and  to  test  the  Mk  54 
torpedo  upgrades.  These  documents  are  out  of  date  and  do  not 
reflect  the  current  threats,  the  current  threat  capabilities,  or  the 
current  or  desired  torpedo  performance. 

•  The  operational  realism  of  the  Mk  54  BUG  testing  from  fleet 
platforms  suffers  from  significant  test  and  safety  limitations 
intended  to  prevent  the  Mk  54  from  hitting  the  manned 
submarine  target  when  it  is  dropped  from  an  aircraft  and 
due  to  time  constraints  for  completing  the  testing.  The 
time  constraints  associated  with  Mk  54  exercise  torpedo 


employment  and  recovery  often  do  not  allow  sufficient  time 
for  fully  operationally  realistic  events. 

Recommendations 

•  Status  of  Previous  Recommendations.  Two  previous 
recommendations  remain  outstanding. 

1.  The  unresolved  IOT&E  of  the  Mk  54  terminal  homing 
is  superseded  by  changes  to  the  Mk  54  BUG  software; 
however,  the  updated  terminal  homing  software  will 
require  a  set-to-hit  testing  evaluation  to  resolve  torpedo 
effectiveness. 

2.  The  Navy  should  continue  to  develop  a  lethality  strategy 
that  includes  the  firing  of  the  Mk  54  against  appropriate 
targets. 

•  FY12  Recommendations.  The  Navy  should: 

1.  Complete  Mk  54  BUG  OT&E  in  2013.  The  testing 
should  include  scenarios  against  representative 
surrogates  employing  current  threats,  tactics,  and  torpedo 
countermeasures. 

2.  Obtain  an  operationally  realistic  mobile  set-to-hit  target  and 
complete  the  terminal  homing  testing  of  the  Mk  54  torpedo. 

3.  Investigate  alternatives,  such  as  the  use  of  a  portable  range, 
to  minimize  or  eliminate  the  test  and  safety  limitations  that 
minimize  operational  realism  in  Mk  54  testing. 
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Mobile  User  Objective  System  (MUOS) 


Executive  Summary 

•  The  Navy  successfully  launched  the  first  Mobile  User 
Objective  System  (MUOS)  satellite  (MUOS-1)  in 
February  2012,  and  conducted  contractor  and  government 
developmental  testing  on  the  MUOS-1  legacy  communications 
capability  and  primary  and  alternate  control  stations. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  a  Multi-Service  Operational  Test  and 
Evaluation  (MOT&E)  from  August  15  -  29,  2012.  Integrated 
developmental  testing  and  preliminary  analysis  of  operational 
testing  indicate  the  MUOS-1  satellite  is  capable  of  providing 
legacy  Ultra-High  Frequency  (UHF)  communications  to 
mobile  users  and  the  Navy  is  able  to  command  and  control 
the  satellite  over  the  MUOS  primary  and  back-up  telemetry, 
tracking,  and  commanding  systems. 

•  Continuing  challenges  integrating  the  secure  MUOS 
waveform  onto  the  Joint  Tactical  Radio  System  (JTRS) 
Handheld,  Manpack,  and  Small  Form  Fit  (HMS)  terminals 
may  cause  cost  increases  and  schedule  delays. 

System 

•  MUOS  is  a  satellite -based  communications  network  designed 
to  provide  worldwide,  narrowband,  beyond  line-of-sight, 
point-to-point,  and  netted  communication  services  to 
multi-Service  organizations  of  fixed  and  mobile  terminal 
users.  The  Navy  designed  MUOS  to  provide  10  times  the 
throughput  capacity  of  the  current  narrowband  satellite 
communications.  The  Navy  intends  for  MUOS  to  provide 
increased  levels  of  system  availability  over  the  current 
constellation  of  UHF  follow-on  satellites,  as  well  as  improved 
availability  for  small,  disadvantaged  terminals. 

•  MUOS  consists  of  six  segments: 

-  The  space  transport  segment  consists  of  four  operational 
satellites  and  one  on-orbit  spare.  Each  satellite  hosts  two 
payloads:  a  legacy  communications  payload  that  mimics 
the  capabilities  of  a  single  UHF  follow-on  satellite,  and  a 
MUOS  communications  payload. 

-  The  Ground  Transport  Segment  is  designed  to  manage 
MUOS  communication  services  and  allocation  of  radio 
resources. 

-  The  Network  Management  Segment  is  designed  to  manage 
MUOS  ground  resources  and  allow  for  government 
controlled  precedence-based  communication  planning. 


-  The  Ground  Infrastructure  Segment  is  designed  to 
provide  transport  of  both  communications  and  command 
and  control  traffic  between  MUOS  facilities  and  other 
communication  facilities. 

-  The  Satellite  Control  Segment  consists  of  MUOS 
Telemetry,  Tracking,  and  Commanding  facilities  at  the 
Naval  Satellite  Operations  Center  Headquarters  and 
Detachment  Delta. 

-  The  User  Entry  Segment  provides  a  MUOS  waveform 
hosted  on  MUOS-compatible  terminals.  The  JTRS  is 
responsible  for  developing  and  fielding  MUOS-compatible 
terminals. 

Mission 

Combatant  Commanders  and  U.S.  military  forces  deployed 
worldwide  will  use  the  integrated  MUOS  satellite 
communications  system  to  accomplish  globally  assigned 
operational  and  joint  force  component  missions  with  increased 
operational  space -based  narrowband,  beyond  line-of-sight 
throughput,  point-to-point,  and  netted  communications  services. 

Major  Contractor 

•  Lockheed  Martin  Space  Systems  -  Sunnyvale,  California 

•  General  Dynamics  C4  Systems  -  Scottsdale,  Arizona 


Activity 

•  The  Navy  successfully  launched  the  MUOS-1  satellite  *  The  Navy  conducted  a  government  Technical  Evaluation  on 

on  February  24,  2012.  The  contractor  conducted  orbital  the  MUOS-1  and  the  Satellite  Control  Segment  July  2  through 

operations  and  contractor  testing  and  transferred  the  MUOS-1  August  1,  2012,  in  preparation  for  operational  testing, 
satellite  to  the  government  Program  Office  on  June  21,  2012. 
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•  COTF  conducted  MOT&E-l  on  the  satellite  control 
capability  and  MUOS-1  legacy  communications  capability 
from  August  15  -  29,  2012,  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

•  The  Navy  completed  MUOS-  2  satellite  thermal  vacuum 
testing  and  is  preparing  to  ship  the  satellite  in  November  2012 
for  an  anticipated  launch  in  July  2013.  The  Navy  plans  to 
ship  satellites  #3,  #4,  and  #5  in  FY13,  FY14,  and  FY15, 
respectively. 

•  The  Navy  operators  have  completed  training  and  are 
commanding  the  MUOS-1  satellite  at  the  Naval  Satellite 
Operations  Center  in  Point  Mugu,  California. 

•  The  ground  transport  site  preparation  at  Wahiawa,  Hawaii; 
Geraldton,  Australia;  and  Northwest,  Virginia,  is  on  schedule. 

•  The  Navy’s  JTRS  Network  Enterprise  Domain  program  is 
working  to  resolve  integration  problems  associated  with 
porting  the  secure  MUOS  waveform  on  the  JTRS  HMS 
Manpack  radio. 

•  The  MUOS  program  developed  an  end-to-end  developmental 
test  strategy  incorporating  additional  testing  prior  to  the 
MUOS  MOT&E-2  and  JTRS  HMS  FOT&E  to  discover  and 
correct  any  integration  problems  prior  to  operational  testing. 

•  The  Navy  plans  to  conduct  the  MUOS  MOT&E-2  in  the  FY14 
timeframe  to  operationally  test  the  full  MUOS  capability. 

Assessment 

•  Although  analysis  of  the  operational  test  data  is  ongoing, 
integrated  developmental  testing  and  preliminary  analysis  of 
operational  testing  suggest  the  MUOS-1  satellite  is  capable  of 
providing  legacy  UHF  communications  to  mobile  users  and 


the  Navy  is  able  to  command  and  control  the  satellite  over  the 
MUOS  primary  and  back-up  systems. 

•  Challenges  integrating  the  secure  MUOS  waveform  onto  the 
JTRS  HMS  terminals  may  cause  cost  increases  and  schedule 
delays  to  both  programs.  If  the  MUOS  and  JTRS  program 
managers  cannot  resolve  latency  and  reliability  issues  due  to 
terminal  processing  constraints,  the  MUOS  MOT&E-2  may  be 
delayed. 

•  COTF  cannot  adequately  test  the  MUOS  capacity  requirements 
in  MOT&E-2  due  to  an  insufficient  number  of  JTRS-equipped 
mobile  users.  COTF  will  need  to  supplement  MOT&E-2 
data  with  validated  modeling  and  simulation  or  other  data  to 
evaluate  the  system’s  ability  to  operate  at  its  planned  capacity 
and  link  availability  levels. 

•  The  MUOS  Performance  Model  that  will  be  used  to  model 
MUOS  capacity  is  behind  schedule  and  needs  improvements 
to  be  accredited  in  time  for  MOT&E-2. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  adequately 
addressed  two  of  the  three  previous  recommendations.  The 
remaining  recommendation  to  operationally  load  the  system 
for  MOT&E-2  is  no  longer  valid. 

•  FY12  Recommendation. 

1.  The  Navy  should  ensure  adequate  resources  and  expertise 
are  applied  to  the  MUOS  Performance  Model  to  make  this  a 
viable  simulation  to  assess  MUOS  capacity  requirements. 


190 


MUOS 


NAVY  PROGRAMS 


MQ-4C  Triton  Broad  Area  Maritime  Surveillance  (BAMS) 


Executive  Summary 

•  DOT&E  approved  the  MQ-4C  Triton  Broad  Area  Maritime 
Surveillance  (BAMS)  Test  and  Evaluation  Master  Plan 
(TEMP)  in  January  2012  to  guide  system  development  and 
operational  testing  through  an  initial  operational  assessment 
and  Milestone  C  Low-Rate  Initial  Production  (LRIP) 
decision. 

•  The  Navy  postponed  MQ-4C  first  flight  and  the  start 
of  developmental  flight  testing  from  May  2012  until  at 
least  January  2013  due  to  aircraft  flight  control  computer 
software  stability  problems.  Resulting  test  schedule 
revisions  compress  developmental  and  operational  test 
events  and  significantly  increase  schedule  risk  prior  to 
the  planned  operational  assessment  in  June  2013  and  the 
Milestone  C  decision  in  October  2013.  The  reduction  in 
planned  developmental  testing  means  that  evaluating  the 
planned  operational  assessment  is  particularly  critical  to  an 
informed  production  decision. 

•  The  Navy  began  the  Multi-Function  Active  Sensor 
(MFAS)  radar  payload  risk-reduction  flight  test  program 
in  December  2011  using  a  Northrop  Grumman  surrogate 
test  bed  aircraft  to  identify  and  resolve  potential  radar 
performance  problems  prior  to  integration  on  the  MQ-4C 
air  vehicle. 

•  In  June  2012,  a  Navy  BAMS-Demonstrator  (BAMS-D) 
aircraft  crashed  during  a  training  mission  near  Patuxent 
River  Naval  Air  Station,  Maryland.  The  aircraft  was  one 
of  five  Air  Force  RQ-4A  Global  Flawk  Block  10  aircraft 
acquired  by  the  Navy  to  provide  interim  support  for  U.S. 
Central  Command  operations  and  develop  unmanned 
maritime  surveillance  tactics  and  doctrine.  This  mishap  did 
not  affect  the  MQ-4C  Triton  development  and  test  program. 

System 

•  The  MQ-4C  Triton  BAMS  unmanned  aircraft  system 
is  an  Intelligence,  Surveillance,  and  Reconnaissance 
system-of-systems  consisting  of  the  high-altitude, 
long-endurance  MQ-4C  air  vehicle,  sensor  payloads,  and 
supporting  ground  control  stations.  The  MQ-4C  system 
is  a  part  of  the  Navy  Maritime  Patrol  and  Reconnaissance 
family-of-systems,  with  capabilities  designed  to 
complement  the  P-8A  Poseidon. 

•  The  MQ-4C  air  vehicle  design  is  based  on  the  Air 
Force  RQ-4B  Global  Flawk  air  vehicle  with  significant 
modifications  that  include  strengthened  wing  structures, 
anti-ice  and  de-icing  systems,  and  an  air  traffic 
de-confliction  and  collision  avoidance  radar  system. 


% 


•  The  MQ-4C  is  equipped  with  the  MFAS  maritime 
surveillance  radar  to  detect,  identify,  and  track  surface 
targets  and  produce  high-resolution  imagery.  The  BAMS 
Electro-Optical  Infrared  Sensor  provides  full  motion  video 
and  still  imagery  of  surface  targets.  An  Electronic  Support 
Measures  system  detects,  identifies,  and  geo-locates  radar 
threat  signals.  An  Automatic  Identification  System  (AIS) 
receiver  permits  the  detection,  identification,  geo-location, 
and  tracking  of  cooperative  vessels  equipped  with  AIS 
transponders. 

•  Onboard  line-of-sight  and  beyond  line-of-sight  datalink 
systems  transmit  sensor  data  from  the  air  vehicle  to  ground 
control  stations  for  dissemination  to  fleet  tactical  operation 
centers  and  intelligence  exploitation  sites. 

Mission 

•  Commanders  use  units  equipped  with  MQ-4C  to  conduct 
maritime  surveillance  operations  and  provide  high-altitude, 
long-endurance  intelligence  collection. 

•  MQ-4C  operators  detect,  identify,  track,  and  assess 
maritime  and  littoral  targets  of  interest  and  collect  imagery 
and  signals  intelligence  information.  Operators  disseminate 
sensor  data  to  fleet  units  to  support  a  wide  range  of  maritime 
missions  to  include  surface  warfare,  intelligence  operations, 
strike  warfare,  maritime  interdiction,  amphibious  warfare, 
homeland  defense,  and  search  and  rescue. 

Major  Contractor 

Northrop  Grumman  Aerospace  Systems,  Battle  Management 

and  Engagement  Systems  Division  -  Bethpage,  New  York 
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Activity 

•  DOT&E  approved  the  MQ-4C  TEMP  in  January  2012  to  guide 
system  development  and  operational  testing  through  an  initial 
operational  assessment  and  Milestone  C  LRIP  decision. 

•  The  Navy  postponed  MQ-4C  first  flight  and  the  start  of 
developmental  flight  testing  from  May  2012  until  at  least 
December  2012  due  to  aircraft  mission  computer  software 
stability  problems. 

•  The  program  also  deferred  development  and  testing  of  some 
air  vehicle  and  sensor  capabilities  until  after  Milestone  C  in 
order  to  reduce  current  test  schedule  pressures.  The  program 
is  currently  assessing  the  potential  effect  of  these  deferrals  on 
post-Milestone  C  test  schedules  leading  to  the  planned  IOT&E 
in  FY15. 

•  In  June  2012,  the  first  MQ-4C  developmental  test  aircraft 
began  initial  ground  test  activities  leading  to  first  flight. 

•  The  Navy  began  MFAS  radar  payload  risk-reduction  flight 
test  program  in  December  2011  using  a  Northrop  Grumman 
surrogate  test  bed  aircraft  to  identify  and  resolve  potential 
radar  performance  problems  prior  to  integration  on  the  MQ-4C 
air  vehicle. 

Assessment 

•  The  MQ-4C  program  experienced  a  significant  test  schedule 
delay  in  2012,  primarily  due  to  aircraft  flight  control  computer 
software  stability  problems  discovered  during  laboratory 
testing.  These  problems  delayed  first  flight  of  the  MQ-4C 
developmental  test  aircraft  from  May  2012  until  at  least 
January  2013.  The  revised  program  test  schedule  compresses 
developmental  and  operational  test  events  and  significantly 
increases  schedule  risk  prior  to  the  planned  operational 
assessment  in  June  2013  and  the  Milestone  C  decision  in 
October  2013.  If  developmental  flight  testing  identifies  any 
significant  air  vehicle,  ground  station,  or  mission  system 
deficiencies,  additional  program  schedule  delays  are  possible. 

•  Following  delivery  of  the  first  flight  test  air  vehicle  in 
June  2012,  acceptance  and  ground  testing  proceeded 


more  slowly  than  expected.  Additional  ground  test  delays 
encountered  in  early  FY13  may  residt  in  further  delays  in  the 
flight  test  program. 

•  The  MFAS  radar  risk-reduction  flight  test  program  progressed 
more  slowly  than  the  Navy  expected  in  FY12.  The  program 
is  currently  focusing  MFAS  flight  test  activities  on  improving 
identified  sensor  software  stability,  maritime  target  detection 
and  tracking  consistency,  and  radar  image  quality  issues. 

•  In  June  2012,  a  Navy  BAMS-D  aircraft  crashed  during  a 
training  mission  near  Patuxent  River  Naval  Air  Station, 
Maryland.  The  aircraft  was  one  of  five  Air  Force  RQ-4A 
Global  Hawk  Block  10  aircraft  acquired  by  the  Navy  to 
provide  interim  support  for  U.S.  Central  Command  operations 
and  develop  unmanned  maritime  surveillance  tactics  and 
doctrine.  BAMS-D  airframe  design,  aircraft  systems,  and 
sensor  payloads  differ  significantly  from  the  more  advanced 
MQ-4C  Triton  system  that  will  begin  flight  test  in  FY13.  This 
mishap  did  not  affect  the  MQ-4C  Triton  development  and  test 
program. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendation. 

1.  The  Navy  should  fully  implement  the  event-driven  test 
and  evaluation  strategy  outlined  in  the  January  2012 
TEMP  to  ensure  that  the  program  completes  the  previously 
approved  system  maturity  demonstrations  and  operational 
assessments  prior  to  a  Milestone  C  LRIP  decision.  Given 
the  reductions  in  developmental  testing  that  are  occurring, 
the  operational  assessment  will  be  a  key,  critical  source 
of  information  on  the  system’s  performance  prior  to  the 
production  decision. 
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Multi-Static  Active  Coherent  (MAC)  System 


Executive  Summary 

•  DOT&E  placed  the  Multi-Static  Active  Coherent  (MAC) 
program  on  oversight  in  2012.  MAC  will  be  the  primary 
wide-area  acoustic  search  system  for  the  P-8A;  without  it, 
P-8A  has  limited  large-area  Anti-Submarine  Warfare  (ASW) 
search  capability. 

•  The  Navy  conducted  developmental  testing  in  2012;  the 
Navy  plans  to  begin  operational  testing  in  early  FY13. 

System 

•  The  MAC  system  is  an  active  sonar  system  composed  of 
two  types  of  buoys  (source  and  receiver)  and  an  acoustic 
processing  software  suite.  It  is  employed  by  the  Navy’s 
maritime  patrol  aircraft  (P-3Cs  and  eventually  P-8As)  to 
search  for  and  locate  threat  submarines  in  a  variety  of  ocean 
conditions. 

•  MAC  is  an  upgrade  to  the  Navy’s  current  Improved  Extended 
Echo  Ranging  (IEER)  system,  which  employs  non-coherent 
sources  to  produce  loud  sounds  that  reflect  off  submarine 
targets;  these  echoes  are  then  detected  by  receiver  buoys. 
MAC  employs  the  same  receiver  buoys,  but  uses  new 
coherent  source  buoys  that  enables  multiple  pings,  optimized 
waveforms,  and  various  ping  durations,  none  of  which  the 
legacy  IEER  system  provided. 

•  The  Navy  intends  to  initially  employ  MAC  on  P-3Cs  in  a 
limited  set  of  acoustic  environments.  Future  increments  of 
MAC  will  be  employed  on  P-8A  and  in  a  wider  variety  of 
acoustic  ocean  environments  in  order  to  span  the  operational 
envelope  of  threat  submarine  operations. 


•  MAC  is  expected  to  have  fewer  effects  on  marine  mammals 
and  the  environment  than  the  legacy  IEER  system. 

•  MAC  will  be  the  primary  wide-area  acoustic  search  system  for 
the  P-8A. 

Mission 

The  Navy  intends  for  P-3C  and  P-8A  crews  equipped  with  MAC 
to  support  the  search,  detect,  and  localization  phases  of  the 
ASW  mission.  MAC  is  particularly  focused  on  large-area  active 
acoustic  searches  for  threat  submarines. 

Major  Contractors 

•  Lockheed  Martin  Corporation  -  Manassas,  Virginia 

•  Sparton  Electronics  Florida,  Inc.  -  De  Leon  Springs,  Florida 

•  Ultra  Electronics,  Undersea  Sensor  Systems  Incorporated 
(USSI)  -  Columbia  City,  Indiana 


Activity 

•  DOT&E  placed  the  MAC  program  on  oversight  in  2012.  The 
Navy  intends  MAC  to  be  a  primary  system  for  the  successful 
ASW  mission  performance  of  the  Maritime  Patrol  P-3  aircraft 
and  the  new  P-8A  aircraft.  Because  of  technical  difficulties 
integrating  multi-static  active  wide-area  ASW  search  systems, 
the  Navy  deferred  the  P-8A  Maritime  Patrol  aircraft’s 
wide-area  search  requirements  and  deferred  this  testing  on 
P-8A. 

•  MAC  achieved  Milestone  C  in  March  2012.  DOT&E  did 
not  approve  the  program’s  Test  and  Evaluation  Master  Plan 
(TEMP)  since  it  was  written  prior  to  DOT&E ’s  oversight; 
however,  DOT&E  accepted  the  current  Navy  TEMP  as 
adequate  for  MAC’S  IOT&E.  An  update  to  the  TEMP 

is  required  to  plan  for  future  MAC  upgrades  and  for  the 
implementation  of  MAC  on  the  P-8A. 

•  The  program  conducted  developmental  and  integrated  testing 
on  P-3C  aircraft  in  the  summer  of  2012;  DOT&E  is  evaluating 
the  data  collected. 


•  The  Navy  reported  the  Active  System  Performance 
Estimate  Computer  Tool  (ASPECT)/Multi-static  Planning 
Acoustics  Toolkit  (MPACT)  used  for  MAC  mission  planning 
inaccurately  predicts  MAC  search  probability  of  detection. 

The  Navy  also  reported  that  the  current  aircraft  system’s 
tactical  mission  software  for  determining  the  position  of 

the  Air  Deployable  Active  Receiver  sonobuoys  and  aircraft 
Mark-on-Top  tactical  procedures  were  not  sufficient  to 
maintain  geographically  accurate  sonobuoy  locations 
throughout  the  MAC  search  field.  The  sonobuoy  location 
inaccuracy  is  caused  by  the  variable  currents  and  resulting 
buoy  drift  rates  encountered  in  the  large  ocean  search  field.  In 
October  2012,  the  Navy  waived  these  deficient  conditions  for 
the  MAC  IOT&E. 

•  The  Navy  will  begin  MAC  operational  testing  on  P-3C 
aircraft  in  early  FY13.  The  Navy  began  test  planning  for  the 
integration  of  MAC  onto  P-8  A  as  well  as  planning  for  the 
testing  of  a  future  upgrade  to  the  MAC  system  in  late  FY12. 
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Assessment 

•  Based  on  the  limited  developmental  testing  data  available 
to  date,  the  reliability  of  the  buoys  and  the  software  appear 
to  be  meeting  developmental  test  requirements.  The  Navy 
and  DOT&E  will  evaluate  final  configuration  changes  to  the 
system  during  IOT&E. 

•  The  IOT&E  will  not  fully  examine  the  capability  of  MAC 
across  all  operational  conditions,  operational  environments, 
and  target  types;  additional  testing  is  necessary  once  MAC 
is  integrated  onto  P-8A  operational  systems  to  examine 
performance  under  other  conditions.  Additional  testing  is  also 
required  to  examine  planned  system  upgrades. 

•  The  inaccurate  ASPECT/MPACT  probability  of  detection 
prediction  tool  used  for  MAC  and  the  inaccurate  sonobuoy 
locating  systems  will  likely  affect  the  MAC  system’s  mission 
performance  and  the  ability  of  the  system  to  accurately  locate 
submarine  targets.  These  inaccuracies  could  result  in  operators 
installing  a  MAC  search  field  that  is  not  optimized  based  on 
the  environmental  conditions  for  detecting  the  target  in  the 
required  search  area.  The  buoy  location  inaccuracies  will 
affect  the  crew’s  ability  to  efficiently  and  accurately  locate  the 


target.  Additional  testing  is  required  to  assess  the  impact  of 
these  waived  conditions. 

•  Based  on  the  initial  scope  of  the  system’s  capabilities, 
the  planned  IOT&E  will  allow  an  assessment  of  MAC 
performance  in  some  threat  areas  and  against  nuclear 
submarine  targets.  DOT&E  requires  future  testing  to 
determine  performance  against  diesel  electric  submarine 
targets  and  in  the  variety  of  threat  environmental  conditions 
where  the  Navy  intends  to  employ  the  MAC  system. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendation. 

1.  Update  the  MAC  TEMP  to  include  sufficient  testing  in 
other  operational  conditions,  operational  environments, 
and  against  diesel-electric  submarines.  The  TEMP  should 
include  planned  testing  on  both  P-3C  and  P-8A. 
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MV-22  Osprey 


Executive  Summary 

•  The  Navy  conducted  FOT&E  in  June  2012. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  evaluated  capabilities  of  the  latest  Block  C  software  and 
six  other  minor  enhancements. 

•  Testing  demonstrated  the  utility  of  all  new  enhancements  with 
the  exception  of  the  Traffic  Advisory  System  (TAS).  Intended 
to  warn  pilots  of  impending  collision  with  approaching 
aircraft,  the  TAS  does  not  distinguish  between  approaching 
aircraft  and  aircraft  in  formation.  Additional  development  is 
needed  to  address  operational  test  findings  and  improve  the 
utility  of  TAS  for  the  MV-22  fleet. 

•  The  Block  C  aircraft  demonstrated  improved  reliability, 
availability,  and  maintainability  compared  to  Block  B  aircraft 
performance  in  previous  operational  tests.  The  MV-22  has 
met  all  RAM  requirements  in  the  Capability  Production 
Document. 

•  The  Navy  continues  to  execute  a  viable  reliability  growth 
program  for  the  MV-22  fleet. 

System 

•  There  are  two  variants  of  the  V-22:  the  Marine  Corps  MV-22 
and  the  Air  Force/U.S.  Special  Operations  Command  CV-22. 
The  air  vehicles  for  Air  Force  and  Marine  Corps  missions 
are  nearly  identical,  with  common  subsystems  and  military 
components  sustainable  by  each  Service’s  logistics  system. 

•  The  Marine  Corps  is  replacing  the  aging  CH-46  and  CF1-53D 
helicopters  with  MV-22s.  The  MV-22  is  a  tilt  rotor  aircraft 
capable  of  conventional  wing-borne  flight  and  vertical  take-off 
and  landing. 

•  The  MV-22  can  carry  24  combat-equipped  Marines  and 
operate  from  ship  or  shore.  It  can  carry  an  external  load  up  to 
10,000  pounds  over  40  nautical  miles  ship-to-shore  and  return, 
and  can  self-deploy  2,267  nautical  miles  with  a  single  aerial 
refueling. 

•  Block  C  (software  version  Cl. 01)  enhancements  include: 

-  Redesigned  Environmental  Control  System  (ECS),  which 
enhances  system  reliability  and  improves  cabin  cooling 
using  directional  nozzles,  larger  heat  exchangers,  and  a 
new  ECS  controller. 

-  Electronic  Standby  Flight  Instruments  (ESFI),  which 
replace  the  analog  standby  instrument  cluster. 

-  Color  Weather  Radar  System,  which  provides  weather 
detection,  ground  mapping,  and  sea  search  functions. 


-  Day  Fleads-up  Display  (F1UD),  which  provides  basic 
flight  and  performance  instrument  information  in  a 
helmet-mounted  monocle. 

-  TAS,  which  is  intended  to  warn  MV-22  pilots  of  other 
aircraft  in  close  proximity  or  with  the  potential  for 
collision. 

-  Cabin  Situational  Awareness  Device,  which  displays 
essential  mission  infonnation  to  troop  commanders  in  the 
cabin  and  assists  the  troop  commander  to  communicate 
with  Marines  independent  of  the  aircrew. 

Mission 

•  Squadrons  equipped  with  MV-22s  will  provide  medium-lift 

assault  support  in  the  following  operations: 

-  Ship-to-Objective  Maneuver 

-  Sustained  operations  ashore 

-  Tactical  recovery  of  aircraft  and  personnel 

-  Self-deployment 

-  Amphibious  evacuation 

•  Currently  deployed  squadrons  are  providing  high-tempo 

battlefield  transportation  in  Iraq  and  Afghanistan. 

Major  Contractors 

Bell-Boeing  Joint  Venture: 

•  Bell  Flelicopter  -  Amarillo,  Texas 

•  The  Boeing  Company  -  Ridley  Township,  Pennsylvania 


Activity 

•  The  Navy  conducted  FOT&E  in  June  2012  in  accordance 
with  the  DOT&E-approved  plan.  This  dedicated  operational 
test  was  preceded  by  about  8  months  of  integrated  testing  in 


which  MV-22  aircraft  accumulated  320  flight  hours.  During 
operational  test,  aircraft  accumulated  approximately  67  flight 
hours. 
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•  Commander,  Operational  Test  and  Evaluation  Force  evaluated 
capabilities  of  the  latest  Block  C  enhancements:  software 
version  C1.01,  upgraded  ECS,  ESFI,  Color  Weather  Radar 
System,  Day  HUD,  TAS,  and  Cabin  Situational  Awareness 
Device  upgrades. 

•  VMX-22  conducted  all  operational  test  missions  at  or  near  the 
Marine  Corps  Air  Station  New  River,  North  Carolina,  using 
four  production-representative  aircraft  (two  Block  C  aircraft 
and  two  Block  B  aircraft).  DOT&E  observed  as  passengers  on 
all  of  the  operational  test  missions. 

Assessment 

•  Except  for  minor  functionality  upgrades,  software  version 
Cl. 01  is  identical  to  the  existing  Block  B  software.  DOT&E 
noted  no  degradation  in  existing  capability  during  testing. 

•  The  upgraded  ECS  proved  effective  at  cooling  troops  in  the 
cabin.  The  adjustable  nozzles  allow  troops  to  direct  cool/warm 
air  onto  their  head/neck.  In  surveyed  responses,  78  percent  of 
Marines  in  a  cabin  with  upgraded  ECS  described  their  general 
comfort  during  missions  as  good  or  excellent  versus  45  percent 
of  those  in  a  cabin  without  the  upgraded  ECS. 

•  The  ESFI  was  well  mechanized  and  integrated  into  the  cockpit 
with  over  75  percent  of  survey  responses  indicating  adequate 
visibility  of  ESFI,  and  about  65  percent  noting  the  ESFI 

was  effective  for  management  of  flight  profile  during  basic 
instrument  maneuvering.  Pilots  noted  a  lag  in  the  Vertical 
Velocity  Indicator  of  about  1  to  5  seconds,  which  makes 
altitude  management  difficult. 

•  The  Color  Weather  Radar  provided  accurate  weather 
information  to  include  precipitation  intensity  and  storm  cell 
turbulence.  The  system  clearly  defined  terrain  features  along 
coastlines  out  to  20  nautical  miles,  and  easily  detected  ships 
in  sea  search  mode.  The  location  and  integration  of  the  Color 
Weather  Radar  was  not  ideal.  The  radar  display  and  controls 
are  accessible  only  to  the  right  seat  pilot,  and  the  pilot  must 
dedicate  one  of  two  multi-functional  displays  to  view  the 
weather  radar. 

•  The  Day  HUD  enhances  the  pilot’s  situational  awareness 
during  tactical  situations  and  reduced  visibility  conditions. 

•  Troop  commanders  said  the  Cabin  Situational  Awareness 
Display  enhances  situational  awareness.  Access  to 
GPS  updates  for  handheld  devices,  way  points,  flight 


plans,  location,  and  other  mission  information  increased 
commanders’  confidence  in  knowing  their  location  on  leaving 
the  aircraft  and  their  ability  to  navigate  the  terrain  once 
disembarked. 

•  Testing  demonstrated  the  lack  of  utility  of  the  TAS.  This 
system  displays  the  top  three  traffic  advisory  priorities  by 
calculating  the  bearing,  range,  and  altitude  of  nearby  aircraft. 
These  top  three  priorities  include  other  aircraft  in  formation, 
essentially  rendering  the  advisory  information  useless  in  most 
MV-22  multi-aircraft  missions. 

•  The  Block  C  aircraft  demonstrated  improved  reliability, 
availability,  and  maintainability  relative  to  aircraft  in  previous 
operational  tests  and  the  MV-22  fleet,  and  met  requirements 
for  reliability,  availability,  and  maintainability. 

•  An  analysis  of  reliability  data  from  2008  to  2012  revealed 
that  integrated  wiring  systems  in  engine  nacelles  have  high 
failure  rates  from  intrusion  of  sand  and  oil,  causing  internal 
chafing  and  deterioration  of  insulated  coatings.  PMA-275  has 
funded  a  program  to  redesign  and  replace  13  wiring  bundles  to 
improve  wiring  system  reliability  and  repairability. 

•  No  additional  flight  testing  or  engineering  analysis  have  been 
done  indicating  a  change  would  be  appropriate  to  DOT&E’s 
September  2005  assessment  that  the  MV-22  cannot  perform 
autorotation  to  a  survivable  landing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
made  improvements  and  has  plans  to  make  other  reliability 
improvements  to  the  icing  protection  system,  as  recommended 
in  FY11.  The  Navy  should  conduct  operational  testing  in  icing 
conditions  when  all  icing  protection  system  enhancements  are 
completed. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Continue  development  and  testing  to  improve  overall 
MV-22  reliability,  availability,  and  maintainability  with 
particular  emphasis  on  the  flight  controls,  integrated  wiring, 
and  drive  train  subsystems. 

2.  Withhold  fielding  the  TAS  as  implemented  in  FOT&E. 
Additional  development  is  needed  to  address  operational 
test  findings  and  improve  the  utility  of  TAS  for  the  MV-22 
fleet. 
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Navy  Multiband  Terminal  (NMT) 


Executive  Summary 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  the  Navy  Multiband  Terminal  (NMT) 
IOT&E  from  July  20  through  August  19,  2011,  on  two  surface 
ships,  one  submarine,  a  shore  site,  and  various  supporting  sites. 

•  The  NMT  met  requirements  to  provide  legacy  Extremely  High 
Frequency  (EHF)  communications  over  Milstar  and  Ultra 
High  Frequency  Follow-On  EHF  Enhanced  payloads;  X-band 
over  the  legacy  Defense  Satellite  Communication  System  and 
Wideband  Global  System  (WGS);  and  Ka-Band  over  WGS. 

•  When  the  IOT&E  showed  that  NMT  is  not  operationally 
suitable,  the  NMT  program  manager  performed  root 
cause  analyses  and  undertook  corrective  actions  to  correct 
deficiencies  found  during  the  IOT&E  and  improve  suitability. 

•  COTF  conducted  further  operational  testing  from  June  1 
through  August  1,  2012,  to  verify  that  the  deficiencies  had 
been  corrected.  The  testing  included  two  surface  ships  and 
one  shore  site,  operating  under  realistic  conditions.  The  NMT 
is  now  operationally  suitable. 

System 

•  The  NMT  system  is  the  next-generation  maritime  military 
satellite  communications  terminal  for  the  Navy  and  its 
coalition  partners;  the  Navy  uses  it  for  enhancing  protected 
and  survivable  satellite  communications. 

•  The  NMT  is  interoperable  with  the  legacy  service  satellite 
communications  terminals,  including  the  Follow-on  Terminal 
and  Navy  EHF  Satellite  Program. 

•  The  NMT  has  variants  for  surface  ships,  submarines,  and  shore 
sites.  The  NMT  system  variants  have  two  major  component 
groups:  the  Communications  Group  and  the  Antenna  Group. 

•  The  Communications  Group  includes  the  following: 

-  Operator  User  Interface 

-  Power  Distribution  Unit 

-  Keyboard 

-  EHF  and  Wideband  drawers 

-  Prime  Power  Interface 

•  The  Antenna  Group  varies  across  different  platforms  and 
includes  new,  reused,  and  modified  antennas  to  support  the 
required  Q-  and  Ka-Bands,  as  well  as  X-band  with  Global 
Broadcasting  System. 


•  The  key  features  of  the  NMT  system  are: 

-  Open  system  architecture 

-  Full  compatibility  with  legacy  terminal  components 

-  High  commonality,  reliability,  and  effective  fault  isolation 

-  Mission  Planning  capability 

Mission 

The  Navy  Component  Commander  uses  the  NMT  to  provide 
secure,  protected,  and  survivable  connectivity  across  the 
spectrum  of  mission  areas  including  land,  air,  and  naval  warfare; 
special  operations;  strategic  nuclear  operations;  strategic  defense; 
theater  missile  defense;  and  space  operations  and  intelligence. 

Major  Contractor 

Raytheon  Net-Centric  Systems  -  Marlboro,  Massachusetts 


Activity 

•  COTF  conducted  the  NMT  IOT&E  from  July  20  through 
August  19,  2011,  on  two  surface  ships,  one  submarine,  a  shore 
site,  and  various  supporting  sites.  COTF  executed  the  test  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan  and  test  plan. 


•  The  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  authorized  the  NMT  Program  Office  to 
procure  FY12  quantities  of  26  NMTs  in  support  of  the 
program  and  an  additional  24  NMTs  in  support  of  other 
programs  such  as  foreign  military  sales. 


NMT 
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•  The  NMT  program  manager  performed  root  cause  analyses 
and  undertook  corrective  actions  to  correct  deficiencies  found 
during  the  IOT&E. 

•  COTF  conducted  further  operational  testing  from  June  1 
through  August  1,  2012,  to  verify  that  the  deficiencies  had 
been  corrected.  The  testing  included  two  surface  ships  and 
one  shore  site,  operating  under  realistic  conditions. 

•  The  Navy  plans  to  conduct  a  Full-Rate  Production  Decision 
Review  for  NMT  in  November  2012. 

•  COTF  plans  to  conduct  an  FOT&E  in  3QFY 14  to  test  the 
NMT’s  ability  to  operate  over  Advanced  EHF  satellites  using 
the  extended  data  rate  waveform. 

Assessment 

•  NMT  can  meet  requirements  to  provide  legacy  EHF 
communications  over  Milstar  and  Ultra  High  Frequency 
Follow-On  EHF  Enhanced  payloads;  X-band  over  the  legacy 
Defense  Satellite  Communication  System  and  WGS;  and 
Ka-Band  over  WGS.  NMT  also  demonstrated  the  capability  to 
receive  Global  Broadcast  System  broadcasts  over  WGS. 

•  The  FY11  IOT&E  demonstrated  that  the  NMT  is  not 
operationally  suitable.  Deployed  ships’  communications  could 
be  hampered  by  poor  reliability  and  long  repair  times  when 
compared  to  the  Navy’s  requirements.  The  time  needed  to 
repair  the  NMT  when  spare  parts  were  unavailable  resulted  in 
long  periods  of  degraded  operations  or  complete  outages  either 
for  a  specific  communications  band  or  for  the  entire  NMT 
system  during  IOT&E. 

•  During  the  recent  operational  test  to  verify  corrections  of 
deficiencies  observed  during  IOT&E,  the  NMT  demonstrated 
a  Mean  Time  Between  Critical  Failure  of  1,461  hours  for 
surface  ship  variants  and  701  hours  for  shore  variants  against 
a  requirement  of  1,400  hours.  The  low  reliability  for  shore 
variants  is  due  to  software  problems  with  the  operator  interface 


that  require  system  reboots  to  restore  the  NMT  to  operation. 
These  reboots  temporarily  disrupted  communications.  The 
operational  consequences  were  mitigated  by  the  availability 
of  redundant  Q-band  terminals  and  the  ability  to  shift 
communications  between  terminals.  The  corrective  actions 
taken  by  the  program  manager  improved  suitability.  The  NMT 
is  now  operationally  suitable. 

•  The  recent  operational  test  demonstrated  that  improvements 
to  NMT  logistic  support  are  still  required  for  deployed  ships. 
The  NMT  was  capable  of  adequately  supporting  shore  sites 
and  ships  in  port.  Off-board  logistics  delay  time  for  a  ship 
deployed  in  the  Mediterranean  was  287  hours  against  a 
requirement  of  100  or  less  hours. 

•  The  Navy  may  not  identify  additional  risks,  other  than  those 
observed  during  the  IOT&E,  until  the  FOT&E  when  Advanced 
EHF  modes  of  operation,  including  the  new  extended  data 
rate  waveform  and  the  new  Mission  Planning  System,  will 

be  tested.  These  capabilities  were  not  evaluated  during  the 
IOT&E  because  they  depend  on  capabilities  being  delivered 
by  the  Advanced  EHF  program  on  a  different  timeline. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
satisfactory  progress  on  all  previous  recommendations. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Synchronize  the  NMT  FOT&E  with  the  Air  Force’s 
Advanced  EHF  Multi-Service  Operational  Test  and 
Evaluation  to  efficiently  use  test  resources. 

2.  Continue  to  explore  methods  to  improve  logistic 
supportability  for  deployed  ships  prior  to  the  FY14 
FOT&E. 

3.  Correct  the  operator  interface  errors  that  are  reducing  shore 
site  reliability. 
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P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA) 


Executive  Summary 

•  The  Navy  delayed  the  P-8A  IOT&E  from  February  2012  to 
September  2012  to  develop  and  test  a  series  of  mission  system 
software  improvements  intended  to  correct  significant  radar, 
electro-optical/infrared  (EO/IR)  sensor,  Electronic  Support 
Measures  (ESM),  and  communications  system  performance 
deficiencies  identified  during  developmental  testing. 

•  In  August  2012,  the  Navy  completed  development  of 
Operational  Flight  Program  (OFP)  Release  9.3  and  approved 
it  as  the  IOT&E  software  baseline.  Developmental  test  results 
indicate  that  OFP  Release  9.3  improved  system  performance, 
particularly  for  the  ESM  sensor.  However,  it  did  not  resolve 

a  number  of  other  deficiencies  that  could  degrade  operational 
effectiveness  for  all  P-8A  missions.  These  deficiencies 
pose  a  high  risk  to  operational  effectiveness,  particularly 
in  the  Intelligence,  Surveillance,  and  Reconnaissance 
(ISR);  Command,  Control,  and  Communication;  and 
Joint  Interoperability  mission  areas.  Unresolved 
sensor  performance  deficiencies  also  affect  the  primary 
Anti-Submarine  Warfare  (ASW)  mission. 

•  P-8  A  ASW  sonobuoy  acoustic  processing  and  torpedo 
carriage/release  systems  continued  to  mature  during 
developmental  test  events,  despite  recurring  system  stability 
and  reliability  problems. 

•  The  Navy’s  decision  to  waive  P-8A  wide-area  acoustic  search 
requirements  for  IOT&E  and  defer  integration  of  a  multi-static 
active  acoustic  capability  until  at  least  FY14  will  limit  ASW 
mission  effectiveness  in  the  near-term.  In  order  to  achieve  full 
ASW  mission  capability  on  the  P-8A  aircraft,  the  Navy  will 
have  to  complete  future  integration  of  the  Multi-Static  Active 
Coherent  broad  area  acoustic  system  on  the  aircraft.  P-8A 
airframe  and  aircraft  subsystem  maturity  improved  during 
developmental  test  events.  Remaining  weather,  take-off,  and 
flight  envelope  restrictions  will  not  significantly  affect  mission 
operations.  However,  main  tank  fuel  overheating  during 
ground  and  low-level  flight  will  limit  ASW  operations  in  hot 
weather  environments. 

•  P-8A  system  reliability  approached  the  minimum  operational 
requirement  of  11.7  Mean  Flight  Hours  Between  Operational 
Mission  Failure  during  the  OFP  Release  9.3  developmental 
test  phase  just  prior  to  IOT&E.  While  promising,  this 
reliability  assessment  was  inconclusive  due  to  the  small 
number  of  operating  hours  observed  with  OFP  Release  9.3 
installed.  The  IOT&E  results  will  provide  a  more  complete 
assessment  of  system  reliability. 

•  The  P-8A  live  fire  test  program  completed  an  initial 
assessment  of  P-8A  vulnerabilities  for  a  range  of  ballistic 
projectiles.  DOT&E  requires  completion  of  remaining 
vulnerability  test  events  on  the  S-l  static  test  article  to  support 
completion  of  the  P-8A  IOT&E  report  in  advance  of  the 

July  2013  Full-Rate  Production  decision. 


•  The  Navy  did  not  release  the  P-8A  On-Board  Inert  Gas 
Generating  System  (OBIGGS)  for  operational  testing  due 
to  serious  system  design  deficiencies  discovered  during 
developmental  testing.  The  program  is  currently  evaluating 
design  changes  and  will  conduct  further  developmental  testing 
in  late  FY12  and  early  FY13. 

System 

•  The  P-8A  Poseidon  Multi-mission  Maritime  Aircraft  (MMA) 
design  is  based  on  the  Boeing  737-800  aircraft  with  significant 
modifications  to  support  Navy  maritime  patrol  mission 
requirements.  It  will  replace  the  P-3C  Orion. 

•  The  P-8A  incorporates  an  integrated  sensor  suite  that  includes 
radar,  electro-optical,  and  electronic  signal  detection  sensors 
to  detect,  identify,  locate,  and  track  surface  targets.  An 
integrated  acoustic  sonobuoy  launch  and  monitoring  system 
detects,  identifies,  locates,  and  tracks  submarine  targets.  The 
P-8A  carries  Mk  54  torpedoes  and  is  currently  integrating 
the  AGM-84  Harpoon  missile  system  to  engage  identified 
submarine  and  surface  targets.  Sensor  systems  also  provide 
tactical  situational  awareness  information  for  dissemination 
to  the  fleet  and  ISR  information  for  exploitation  by  the  joint 
intelligence  community. 

•  The  P-8A  aircraft  incorporates  aircraft  survivability 
enhancement  and  vulnerability  reduction  systems.  An 
integrated  infrared  missile  detection  system,  flare  dispenser, 
and  directed  infrared  countermeasure  system  is  designed  to 
improve  survivability  against  infrared  missile  threats.  On  and 
off-board  sensors  and  datalink  systems  are  used  to  improve 
tactical  situational  awareness  of  radio  frequency  missile  threat 
systems.  Fuel  tank  inerting  and  fire  protection  systems  reduce 
aircraft  vulnerability. 
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Mission 

•  Theater  Commanders  primarily  use  units  equipped  with  the 
P-8A  MMA  to  conduct  Anti-Submarine  Warfare.  P-8A  units 
detect,  identify,  track,  and  destroy  submarine  targets. 

•  Additional  P-8A  maritime  patrol  missions  include: 

-  Anti-Surface  Warfare  operations  to  detect,  identify,  track, 
and  destroy  enemy  surface  combatants  or  other  shipping 
targets. 

-  Maritime  and  littoral  1SR  operations  to  collect  and 
disseminate  imagery  and  signals  information  for 
exploitation  by  the  joint  intelligence  community. 


-  Collection  and  dissemination  of  tactical  situation 
information  to  improve  the  fleet  common  operational 
picture. 

-  Identification  and  precise  geo-location  of  targets  ashore  to 
support  fleet  strike  warfare  missions. 

Major  Contractor 

Boeing  Defense,  Space,  and  Security  -  St.  Louis,  Missouri 


Activity 

•  The  Navy  delayed  the  P-8A  IOT&E  from  February  2012  to 
September  2012  to  develop  and  test  a  series  of  mission  system 
software  improvements  intended  to  correct  significant  radar, 
EO/IR  sensor,  ESM,  and  communications  system  performance 
deficiencies  identified  during  developmental  testing. 

•  From  May  2012  to  September  2012,  Navy  operational  testers 
conducted  a  series  of  pre -IOT&E  integrated  developmental 
and  operational  test  events  using  production-representative 
test  aircraft  and  interim  OFP  software  releases  to  evaluate 
P-8A  deployment  capabilities  and  assess  evolving  aircraft 
and  mission  system  maturity.  These  events  also  provided 
additional  flight  crew  and  maintenance  training  experience  in 
preparation  for  IOT&E.  Major  integrated  test  events  included: 

-  Joint  Warrior  fleet  exercise  conducted  with  the  United 
Kingdom  and  other  NATO  countries  in  April  2012.  Test 
crews  completed  6  missions  totaling  36.3  flight  hours 
during  this  exercise. 

-  U.S. -Australian  fleet  exercise  in  Australia  in  June  2012. 
Operational  test  crews  completed  6  missions  totaling 
24.8  flight  hours  during  this  exercise. 

-  Rim  of  the  Pacific  fleet  exercise  in  Hawaii  in  July  2012. 
Operational  test  crews  completed  20  missions  totaling 
102.2  flight  hours  during  this  exercise. 

-  Mk  54  torpedo  tests  at  the  Atlantic  Underwater  Test  Center 
in  May  2012,  Cape  Cod  Atlantic  test  areas  in  August  2012, 
and  Hawaii  Pacific  test  ranges  in  September  2012. 

•  In  August  20 12,  the  Navy  completed  development  of 
OFP  Release  9.3  and  approved  it  for  use  in  IOT&E.  This 
release  addressed  some  of  the  most  serious  sensor  performance 
and  communications  system  deficiencies  identified  during 
earlier  testing. 

•  The  Navy  entered  IOT&E  beginning  with  participation  in 
the  U.S.  Pacific  Command  Valiant  Shield  exercise  in  early 
September  2012.  The  Navy  plans  to  complete  IOT&E 

in  December  2012.  The  Navy  is  conducting  testing  in 
accordance  with  the  DOT&E-approved  test  plan. 

•  The  P-8A  live  fire  test  program  completed  ballistic  testing  of 
P-8A  emergency  oxygen  bottles  on  a  surrogate  Boeing  737 
test  aircraft  in  March  2012.  The  program  also  completed  an 


evaluation  of  wing  fuel  tank  tolerance  against  threat-induced 
hydrodynamic  ram  damage  in  September  2012. 

•  The  Navy  completed  an  initial  flight  test  of  the  missile  warning 
system,  flare  dispenser,  and  directed  infrared  countermeasures 
system  against  simulated  infrared  threats.  The  program  is 
planning  to  conduct  additional  hardware-in-the-loop  testing  of 
these  systems  in  early  FY13  to  complete  required  live  fire  and 
operational  test  requirements. 

Assessment 

•  Developmental  test  results  indicate  P-8A  airframe  and  aircraft 
subsystem  maturity  continued  to  improve  as  the  developmental 
test  program  progressed.  Remaining  aircraft  weather,  take-off, 
and  flight  envelope  restrictions  should  not  significantly  affect 
mission  operations  and  are  on  track  for  resolution  prior  to 
P-8A  operational  deployment.  However,  developmental 
testing  identified  main  tank  fuel  overheating  during  ground 
and  low-level  flight  operations  as  a  serious  deficiency  that 
will  limit  ASW  mission  on-station  time  in  hot  weather 
environments. 

•  Developmental  test  results  indicate  that  OFP  Release  9.3 
improved  system  performance,  particularly  for  the  ESM 
sensor.  However,  it  did  not  resolve  a  number  of  other 
deficiencies  that  could  seriously  degrade  operational 
effectiveness  for  all  P-8A  missions.  These  deficiencies  pose 
a  high  risk  to  operational  effectiveness,  particularly  in  the 
ISR;  Command,  Control,  and  Communication;  and  Joint 
Interoperability  mission  areas: 

-  Common  Data  Link  and  International  Maritime  Satellite 
data  transfer  deficiencies  that  prevent  reliable  transmission 
of  synthetic  aperture  radar  (SAR)  and  EO/IR  imagery 
intelligence  products  to  operational  users 

-  Ineffective  voice  satellite  communications  systems  that 
prevent  transmission  and  receipt  of  mission  critical 
information 

-  SAR  high-resolution  image  quality  problems  that  degrade 
imagery  intelligence  capabilities 

-  Radar  pointing  errors  that  prevent  effective  SAR  imagery 
collection  for  some  littoral/land  targets 
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-  EO/IR  sensor  cross-cue  errors  and  inoperative  auto-track 
modes  that  significantly  increase  sensor  operator  workload 

-  Non-standard  imagery  metadata  formats  that  prevent  joint 
intelligence  exploitation  of  P-8A  imagery  intelligence 
products 

-  Ineffective  ESM  Specific  Emitter  Identification  subsystem 
that  does  not  reliably  collect  and  identify  emitter  signatures 
to  enable  identification  of  specific  hostile  targets 

-  Radar  track-while-scan  deficiencies  that  degrade  P-8A  and 
fleet  situational  awareness 

•  P-8  A  AS  W  sonobuoy  launch,  positioning,  and  acoustic 
processing  systems  continued  to  mature  during  developmental 
test  events,  despite  recurring  system  stability  and  reliability 
problems.  Unresolved  radar,  EO/IR,  and  ESM  sensor 
performance  deficiencies  also  affect  ASW  mission  operations. 
Excessive  ESM  system  interference  and  bleed  over  onto 
ASW  operator  displays  frequently  preclude  ESM  operation 
during  ASW  operations.  In  addition,  developmental  test 
events  included  few  ASW  search  and  detect  events  against 
operationally  realistic  submarine  targets,  which  increases 

the  potential  for  discovery  of  additional  deficiencies  during 
IOT&E. 

•  The  Navy  decision  to  waive  P-8A  wide-area  acoustic  search 
requirements  for  IOT&E  and  defer  integration  of  a  multi-static 
active  acoustic  capability  until  at  least  FY14  will  limit  ASW 
mission  effectiveness  in  the  near-term.  Future  integration  of 
the  Mrdti-Static  Active  Coherent  broad  area  acoustic  system 
will  be  required  to  achieve  full  ASW  mission  capabilities. 

•  Developmental  test  results  indicate  that  P-8A  torpedo 
release  and  water  impact  point  accuracies  meet  operational 
requirements.  Complete  end-to-end  torpedo  employment 
effectiveness  will  be  assessed  during  IOT&E  events. 

However,  inadequate  weapons  bay  heating  systems  currently 
restrict  torpedo  carriage  altitudes  and  combat  mission  radius 
in  cold  weather  environments.  The  program  is  planning  to 
develop  and  test  system  design  changes  to  improve  weapons 
bay  heating  in  FY13,  prior  to  operational  fielding. 

•  P-8A  system  reliability  improved  significantly  during  the 
final  OFP  Release  9.3  developmental  test  phase  prior  to 
IOT&E,  approaching  the  minimum  operational  requirement  of 
11.7  Mean  Flight  Hours  Between  Operational  Mission  Failure. 
While  promising,  this  pre-IOT&E  reliability  assessment  was 
inconclusive  due  to  the  small  number  of  operating  hours 
observed  with  OFP  Release  9.3  installed.  Most  P-8A  mission 
reliability  failures  are  directly  attributable  to  system  software 
deficiencies.  Hardware  reliability  exceeded  the  1.25  Mean 


Flight  Hour  Between  Operational  Mission  Failure  requirement 
throughout  the  developmental  and  integrated  test  phases. 

•  The  P-8A  live  fire  test  program  completed  an  initial  assessment 
of  P-8A  vulnerabilities  for  a  range  of  ballistic  projectiles. 
DOT&E  requires  completion  of  remaining  vulnerability  test 
events  on  the  S- 1  static  test  article  to  support  completion  of 
the  P-8AIOT&E  report  in  advance  of  the  July  2013  Full-Rate 
Production  (FRP)  decision.  The  current  S-l  test  schedule 
supports  a  live  fire  vulnerability  assessment  in  3QFY13,  but 
with  little  margin  for  additional  delay  prior  to  the  planned  July 
2013  FRP  decision. 

•  The  Navy  did  not  release  the  P-8A  OBIGGS  for  operational 
testing  during  IOT&E  due  to  serious  system  design 
deficiencies  discovered  during  developmental  testing.  The 
program  is  currently  implementing  design  changes  and  has 
begun  further  developmental  testing,  which  will  continue  into 
FY13.  This  system  is  a  critical  P-8A  survivability  feature  that 
maintains  inert  fuel  tank  environments  to  improve  ballistic 
projectile  protection. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  made 
progress  on  all  FY11  recommendations.  Developmental 
testing  cleared  the  P-8A  operational  flight  envelope  to  support 
IOT&E  operational  flight  profiles.  The  program  realigned 
S-l  live  fire  test  schedules  to  support  completion  of  testing 
prior  to  the  FRP  decision.  The  Navy  corrected  many,  but  not 
all,  mission  critical  software  deficiencies  during  the  FY12 
extended  developmental  test  period  leading  to  IOT&E. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Accelerate  correction  of  the  remaining  unresolved  radar, 
EO/IR,  ESM,  and  communications/data  transfer  deficiencies 
and  conduct  operational  testing  to  verify  software  fix 
effectiveness. 

2.  Correct  weapons  bay  heating  and  main  tank  fuel 
overheating  hardware  deficiencies  and  conduct  testing  to 
verify  unrestricted  flight  envelopes  in  cold  and  hot  weather 
environments. 

3.  Closely  monitor  progress  of  live  fire  vulnerability  and 
OBIGGS  test  events  to  ensure  completion  and  data  delivery 
in  time  to  support  DOT&E’s  P-8A  IOT&E  report  and  FRP 
Decision. 

4.  Complete  Test  and  Evaluation  Master  Plan  development 
and  test  planning  for  P-8  A  Increment  1  FOT&E  events 
and  the  series  of  P-8A  Increment  2  developmental  and 
operational  tests  scheduled  to  begin  in  FY 14. 
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Ship  Self-Defense 


Executive  Summary 

•  The  ship  self-defense  mission  for  aircraft  carriers,  destroyers, 
and  amphibious  warfare  ships  coordinates  several  legacy 
shipboard  systems,  as  well  as  five  major  acquisition 
programs:  Ship  Self-Defense  System  (SSDS),  Rolling 
Airframe  Missile  (RAM),  Evolved  SeaSparrow  Missile 
(ESSM),  Cooperative  Engagement  Capability  (CEC),  and  the 
Air  and  Missile  Defense  Radar  (AMDR).  These  comprise 

a  self-defense  capability  for  in-service  ships,  as  well  as  the 
LPD-17,  LHA-6,  DDG-51  Flight  III,  and  CVN-78  ship 
classes  still  in  acquisition. 

•  DOT&E  issued  a  classified  report  to  Congress  in 
November  2012  entitled  “Ship  Self-Defense  Operational 
Mission  Capability  Assessment  Report.” 

•  While  the  integration  of  sensor  and  weapon  systems  with 
the  command  and  decision  system  enhances  the  ships’ 
self-defense  capability  over  non-integrated  combat  systems, 
the  Navy  has  not  successfully  demonstrated  the  ability  to 
effectively  complete  the  self-defense  mission  against  the 
types  of  threats  and  threat  scenarios  for  which  the  overall 
system  was  designed. 

•  The  Navy  must  complete  the  currently  planned  operational 
test  program  and  conduct  additional  testing  to  demonstrate 
the  correction  of  significant  deficiencies  with  SSDS  Mk  2, 
RAM,  ESSM,  CEC,  and  legacy  ship  self-defense  combat 
system  elements. 

System 

Surface  ship  self-defense  is  addressed  by  several  legacy  combat 
system  elements  (ship  class-dependent)  and  five  acquisition 
programs:  SSDS,  RAM,  ESSM,  CEC,  and  AMDR. 

SSDS 

•  SSDS  is  a  local  area  network  that  uses  open  computer 
architecture  and  standard  Navy  displays  to  integrate  a 
surface  ship’s  sensors  and  weapons  systems  to  provide 
an  automated  detect-track-engage  sequence  for  ship 
self-defense.  SSDS  Mk  1  is  the  command  and  control 
system  for  LSD-4 1/49  class  ships. 

•  SSDS  Mk  2  has  six  variants: 

-  Mod  1,  used  in  CVN-68  class  aircraft  carriers 

-  Mod  2,  used  in  LPD-17  class  amphibious  ships 

-  Mod  3,  used  in  LHD-7/8  class  amphibious  ships 

-  Mod  4,  in  development  for  LHA-6  class  amphibious 
ships 

-  Mod  5,  in  development  for  LSD-41/49  class 
amphibious  ships 

-  Mod  6,  in  development  for  CVN-78  class  aircraft 
carriers 

RAM 

•  The  RAM,  jointly  developed  by  the  United  States  and 
the  Federal  Republic  of  Germany,  provides  a  short-range, 
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lightweight,  self-defense  system  to  defeat  Anti-Ship  Cruise 
Missiles  (ASCMs).  RAM  is  currently  installed  in  all 
aircraft  carriers  and  amphibious  ships  (except  LPD-4  class). 

•  There  are  four  RAM  variants: 

-  RAM  Block  0  uses  dual  mode,  passive  radio 
frequency /infrared  guidance. 

-  RAM  Block  1  adds  infrared  guidance  improvements  to 
extend  defense  against  non-radio-frequency-radiating 
ASCMs. 

-  RAM  Block  1A  extends  the  capability  of  RAM  Block  1 
against  non-ASCM  targets  including  helicopters,  slow 
aircraft,  and  surface  threats. 

-  RAM  Block  2  is  in  development  and  will  extend  the 
capability  of  RAM  Block  1A  against  newer  classes  of 
ASCM  threats. 

ESSM 

•  The  ESSM,  cooperatively  developed  among  13  nations, 
is  a  medium-range,  ship-launched  self-defense  guided 
missile  designed  to  defeat  ASCM,  surface,  and  low 
velocity  air  threats.  The  ESSM  is  currently  installed 
on  DDG-51  Flight  IIA  destroyers,  as  well  as  CVN-68 
class  aircraft  carriers  equipped  with  the  SSDS  Mk  2 
Mod  1  Combat  System.  The  Navy  is  planning  for  future 
ESSM  installations  in  CG-47  class  cruisers,  LHA-6  class 
amphibious  assault  ships,  CVN-78  class  aircraft  carriers, 
DDG-1000  class  destroyers,  and  DDG-51  Flight  III  class 
destroyers. 

•  There  are  two  variants  of  ESSM. 

-  ESSM  Block  1  is  a  semi-active  radar-guided  missile  that 
is  currently  in-service. 

-  ESSM  Block  2  is  in  development  and  will  have 
semi-active  radar-guidance  as  well  as  active 
radar-guidance. 
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CEC 

•  CEC  is  a  sensor  network  with  integrated  fire  control 
capability  that  is  intended  to  significantly  improve  battle 
force  air  and  missile  defense  capabilities  by  combining 
data  from  multiple  battle  force  air  search  sensors  on 
CEC-equipped  units  into  a  single,  real-time,  composite 
track  picture.  The  two  major  hardware  pieces  are  the 
Cooperative  Engagement  Processor,  which  collects  and 
fuses  radar  data,  and  the  Data  Distribution  System,  which 
exchanges  the  Cooperative  Engagement  Processor  data. 
CEC  is  an  integrated  component  of,  and  serves  as  the 
primary  air  tracker  for,  SSDS  Mk  2-equipped  ships. 

•  There  are  four  major  variants  of  CEC: 

-  The  CEC  USG-2  is  used  in  selected  Aegis  cruisers 
and  destroyers,  LPD-17/LHD  amphibious  ships,  and 
CVN-68  class  aircraft  carriers. 

-  The  CEC  USG-2A,  an  improved  version  of  the  USG-2, 
is  used  in  selected  Aegis  cruisers  and  destroyers. 

-  The  CEC  USG-3  is  used  in  the  E-2C  Hawkeye  2000 
aircraft. 

-  The  CEC  USG-3B  is  in  development  for  use  in  the  E-2D 
Advanced  Hawkeye  aircraft. 

AMDR 

•  The  AMDR  is  the  Navy’s  next  generation  radar  system  that 
is  being  developed  to  provide  DDG-51  Flight  III  Destroyer 
combat  systems  with  simultaneous  sensor  support  of 


ballistic  missile  defense  (BMD)  and  air  defense  (AD)  (to 
include  self-defense)  missions. 

Mission 

Naval  Component  Commanders  use  SSDS,  RAM,  ESSM,  and 

CEC,  as  well  as  many  legacy  systems,  to  provide  faster,  more 

effective  accomplishment  of  ship  self-defense  missions. 

•  Naval  surface  forces  use  SSDS  to  provide  automated  and 
integrated  detect-to-engage  ship  self-defense  capability  against 
ASCM,  air,  and  surface  threats. 

•  Naval  surface  forces  use  RAM  to  provide  a  short-range  hard 
kill  engagement  capability  against  ASCM  threats. 

•  Naval  surface  forces  use  ESSM  to  provide  a  medium-range 
hard  kill  engagement  capability  against  ASCM,  surface,  and 
low  velocity  air  threats. 

•  Naval  surface  forces  use  CEC  to  provide  accurate  air  and 
surface  threat  tracking  data  to  SSDS. 

•  Naval  surface  forces  will  use  AMDR  as  a  primary  sensor  for 
simultaneous  BMD  and  AD  (to  include  self-defense)  missions. 

Major  Contractors 

•  SSDS:  Raytheon  -  San  Diego,  California 

•  RAM  and  ESSM:  Raytheon  -  Tucson,  Arizona 

•  CEC:  Raytheon  -  St.  Petersburg,  Florida 


Activity 

•  DOT&E  issued  a  classified  report  to  Congress  on  the  ship 
self-defense  mission  area  in  November  2012.  The  report 
covers  ship  self-defense  related  operational  testing  conducted 
from  February  2008  through  December  2011  aboard  USS 
Ronald  Reagan  (CVN-76),  USS  Carl  Vinson  (CVN-70),  and 
the  Self-Defense  Test  Ship  (SDTS). 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  completed  FOT&E  testing  of  ESSM,  RAM, 
CEC,  and  SSDS  on  the  SDTS  in  December  2011.  Testing 
was  conducted  in  accordance  with  a  DOT&E-approved  test 
plan. 

•  COTF  continued  planning  for  operational  testing  of  the 
ship  self  defense  mission  area  during  IOT&E  of  the  RAM 
Block  2  and  FOT&E  of  the  SSDS  Mk  2  Mod  4  and  ESSM 
on  the  SDTS.  The  Navy  plans  to  continue  testing  in 
November  2012. 

•  COTF  continued  planning  for  IOT&E  testing  of  the  LHA-6 
class  ship  self-defense  combat  system  on  the  SDTS.  The 
Navy  plans  to  commence  IOT&E  testing  in  November  2012. 

Assessment 

•  The  November  2012  DOT&E  ship  self-defense  mission  area 
report  includes  the  following  assessments: 

-  The  CVN-68  ship  class  combat  systems  continue  to 
have  difficulty  defeating  certain  ASCM  raid  types.  In 
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particular,  the  legacy  combat  system  sensor  elements  have 
limited  capability  against  the  threat  surrogates  used  in 
those  raid  types. 

-  The  CVN-68  ship  class  combat  system  continues  to 
have  several  problems  that  hinder  it  from  successfully 
completing  the  ship  self-defense  mission.  Specific 
problems  include  deficiencies  in  weapon  employment 
timelines,  sensor  coverage,  system  track  management, 
and  NATO  ESSM  performance,  as  well  as  deficiencies 
with  the  recommended  engagement  tactics  for  use  against 
multiple  ASCM  threat  classes. 

•  The  test  infrastructure  is  inadequate  to  support  self-defense 
testing  on  the  next  flight  of  destroyers.  There  is  no 
unmanned,  at-sea  test  capability  to  safely  demonstrate  a 
self-defense  capability  for  Aegis  destroyers  against  anti-ship 
missile  threats.  The  test  capability  must  be  in  place  by 
2020  to  support  DDG-51  Flight  III  Destroyer  Combat 
System,  ESSM  Block  2,  and  AMDR  integration  self-defense 
operational  testing. 

•  The  classified  November  2012  DOT&E  report  to  Congress 
contains  further  ship  self-defense  mission  area  assessments. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy 
has  satisfactorily  completed  the  majority  of  previous 
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recommendations.  The  Navy  has  not  resolved  the  following 

previous  recommendations: 

1.  Optimize  SSDS  Mk  2  weapon  employment  timelines  to 
maximize  weapon  probability  of  kill. 

2.  Acquire  range-safe  supersonic  sea-skimming  ASCM 
surrogate  targets  for  ESSM  FOT&E  with  the  Aegis  Combat 
System. 

3.  Ensure  availability  of  a  credible  open-loop  seeker  subsonic 
ASCM  surrogate  target  for  ship  self-defense  combat  system 
operational  tests. 

4.  Correct  the  identified  SSDS  Mk  2  software  reliability 
deficiencies. 

5.  Correct  the  identified  SSDS  Mk  2  training  deficiencies. 

6.  Develop  and  field  deferred  SSDS  Mk  2  interfaces  to  the 
Global  Command  and  Control  System-Maritime  and  the 
TPX-42A(V)  command  and  control  systems. 

7.  Continue  to  implement  the  Program  Executive  Office 
for  Integrated  Warfare  Systems’  plan  for  more  robust, 
end-to-end  systems  engineering  and  associated 
developmental/operational  testing  of  ship  self-defense 
combat  systems. 

8.  Provide  a  capability  to  launch  a  raid  of  four  supersonic  sea 
skimming  targets  at  the  Naval  Air  Warfare  Center/Weapons 
Division,  Point  Mugu,  California,  test  range  to  support  Test 
and  Evaluation  Master  Plan-approved  Air  Warfare/Ship 
Self-Defense  Enterprise  testing  planned  for  FY16. 

9.  Improve  the  ability  of  legacy  ship  self-defense  combat 
system  sensor  elements  to  detect  threat  surrogates  used  in 
specific  ASCM  raid  types. 


10.  Ensure  availability  of  adequate  and  credible  target  resources 
for  ship  self-defense  and  electronic  warfare  operational 
testing. 

1 1 .  Take  action  on  the  classified  recommendations  contained 
in  the  March  2011  DOT&E  report  to  Congress  on  the  ship 
self-defense  mission  area. 

FY12  Recommendations.  Based  on  the  classified  information 

contained  in  the  November  2012  report  to  Congress,  the  Navy 

should: 

1.  Improve  the  SSDS  Mk  2  integration  with  the  Mk  9  Track 
Illuminators  to  better  support  ESSM  engagements,  as 
well  as  preventing  the  Mk  9  Track  Illuminators  from 
contributing  to  the  composite  track  during  certain  threat  raid 
types. 

2.  Develop  combat  system  improvements  to  increase  the 
likelihood  that  ESSM  and  RAM  will  home  on  their 
intended  targets. 

3.  Conduct  additional  operational  testing  on  the  CVN-68 
class  once  the  ship  is  equipped  with  additional  self-defense 
weapons.  This  additional  testing  will  determine  whether 
the  additional  weapons  are  sufficient  to  meet  the  ship’s 
self-defense  requirements. 

4.  Develop  an  unmanned,  at-sea  self-defense  test  capability 
that  will  allow  safe  demonstration  of  the  self-defense 
mission  of  DDG-51  Flight  III  destroyers  against  anti-ship 
missile  threats. 

5.  Take  action  on  the  classified  recommendations  contained  in 
the  November  2012  DOT&E  report  to  Congress  on  the  ship 
self-defense  mission  area. 
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SSN  774  Virginia  Class  Submarine 


Executive  Summary 

•  The  Navy  conducted  two  Virginia  class  FOT&E  events  that 
began  in  FY11  and  extended  into  FY12.  The  first  test  event 
examined  the  Virginia  class  submarine’s  ability  to  operate 
under-ice  and  to  conduct  Anti-Submarine  Warfare  (ASW)  in 
the  Arctic.  The  second  test  event  examined  the  modernized 
Virginia  class  submarine’s  performance  with  the  Navy’s  latest 
combat  system  and  sonar  suite. 

•  DOT&E  issued  a  classified  report  on  the  modernized 
Virginia  FOT&E  in  November  2012,  and  concluded  that  the 
modernization  of  the  combat  system  and  sonar  suite  did  not 
change  the  performance  of  the  Virginia  class  for  the  missions 
tested. 

•  The  Navy  completed  an  update  to  the  Test  and  Evaluation 
Master  Plan  (TEMP)  and  DOT&E  approved  the  revision  on 
July  24,  2012.  The  revision  outlines  plans  to  test  deferred 
capabilities,  electronic  systems  upgrades,  and  affordability 
changes  included  in  the  third  increment  of  submarines 
(Block  III). 

System 

•  The  Virginia  class  submarine  is  the  replacement  for  the  aging 
fleet  of  Los  Angeles  class  submarines.  The  Virginia  class: 

-  Is  capable  of  targeting,  controlling,  and  launching  Mk  48 
Advanced  Capability  torpedoes,  Tomahawk  cruise 
missiles,  and  future  mines 

-  Has  mission  capability  similar  to  the  Seawolf  submarine 
class  with  improvements  to  the  electronic  support  suite, 
sonar,  and  combat  control  systems 

-  Has  a  new-design  propulsion  plant  incorporating 
components  from  previous  submarine  classes 

-  Uses  a  modular  design  and  significant  commercial 
off-the-shelf  computer  technologies  and  hardware  intended 
to  allow  for  rapid  and  cost-effective  technology  refresh 
cycles 

•  The  Navy  is  procuring  and  upgrading  Virginia  class 
submarines  incrementally  in  a  series  of  blocks. 

-  Block  I  (hulls  1-4)  and  Block  II  (hulls  5-10)  ships 
incorporated  the  initial  design  of  the  Virginia  class. 

-  Block  III  (hulls  11-18)  ships  will  include  the  following 
affordability  enhancements: 

■  A  Large  Aperture  Bow  array  will  replace  the  spherical 
array  in  the  front  of  the  ship. 


■  Two  Virginia  payload  tubes  will  replace  the  12  vertical 
launch  tubes.  Each  payload  tube  is  capable  of  storing 
and  launching  six  Tomahawk  land  attack  missiles  used  in 
strike  warfare. 

-  The  Navy  has  not  finalized  the  design  for  Block  IV  and 
beyond  ships. 

Mission 

The  Operational  Commander  will  employ  the  Virginia  class 
submarine  to  conduct  open  ocean  and  littoral  covert  operations  in 
support  of  the  following  submarine  mission  areas: 

•  Strike  Warfare  (STW) 

•  Anti-Submarine  Warfare  (ASW) 

•  Intelligence,  Surveillance,  and  Reconnaissance  (ISR); 
Indications  and  Warnings  (I&W);  and  Electronic  Warfare 
(EW) 

•  Anti-Surface  Ship  Warfare  (ASUW) 

•  Naval  Special  Warfare  (NSW) 

•  Mine  Warfare  (MIW) 

•  Battle  Group  Operations  (BGO) 

Major  Contractors 

•  General  Dynamics  Electric  Boat  -  Groton,  Connecticut 

•  Huntington  Ingalls  Industries,  Newport  News 
Shipbuilding  -  Newport  News,  Virginia 


Activity 

•  In  accordance  with  DOT&E-approved  test  plans,  two  Virginia 
class  submarine  FOT&E  events  started  in  FY11  and  extended 
into  FY12  due  to  data  processing,  data  analysis,  and  reporting 
delays. 


The  Navy  conducted  the  first  FOT&E  period  in 
conjunction  with  a  bi-annual  Ice  Exercise  (ICEX-11). 
This  event  allowed  testers  to  examine  the  Virginia  class 
submarine’s  ability  to  transit  to  and  operate  under-ice  and 
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in  the  Arctic.  This  test  period  also  assessed  the  Virginia 
class’s  ASW  mission  performance  against  a  modern 
threat  SSN  surrogate  under-ice  in  the  Arctic.  As  part 
of  the  transit  to  northern  latitudes,  testers  examined  the 
Virginia  class’  susceptibility  to  fixed  passive  sonar  arrays. 
DOT&E  plans  to  issue  a  report  on  this  testing  in  2QFY13. 
-  The  second  FOT&E  period  was  a  series  of  test  events  to 
examine  the  mission  performance  changes  as  a  result  of 
the  modernization  of  the  Virginia  class  submarines’  sonar 
and  combat  control  system.  These  tests  were  combined 
with  the  operational  evaluations  of  the  latest  variants  of 
the  Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion 
(A-RCI)  Sonar  System,  the  AN/BYG-1  Combat  Control 
System,  and  the  Mk  48  Advanced  Capability  torpedo. 
DOT&E  issued  a  combined  operational  test  report  on  this 
testing  in  November  2012. 

•  The  Virginia  TEMP  also  required  FOT&E  in  order  to 
complete  testing  of  the  Naval  Special  Warfare  mission 
support  capabilities  for  the  Dry  Deck  Shelter  (DDS)  and  for 
a  diver  emergency  recompression  capability  in  the  Virginia 
class  submarine’s  Lock-Out  Trunk  (LOT).  The  Navy 
completed  LOT  and  DDS  system  design  changes  and  began 
test  planning  for  the  LOT  diver  oxygen  recompression  system 
certification  and  for  operations  with  a  DDS  in  FY12.  The 
Navy  plans  to  complete  these  test  events  in  early  FY13. 

•  DOT&E  approved  Revision  G  to  the  TEMP  on  July  24,  2012. 
The  revision  outlines  plans  to  test  the  mission  performance  of 
the  Virginia  class  affordability  changes  (new  bow  array  and 
vertical  payload  tubes)  with  the  electronic  systems  upgrades 
planned  for  the  third  increment  submarines  (Block  III)  and  to 
complete  the  testing  of  deferred  mission  capabilities. 

•  Because  Navy  security  rules  prevent  collection  of  useful 
operational  test  data  from  Virginia  when  conducting  exercises 
with  foreign  ASW  capable  platforms,  the  Navy  finished 
IOT&E  and  recent  FOT&E  without  testing  the  Virginia  class 
submarine  against  one  of  its  primary  threats,  the  foreign 
diesel  electric  submarine  (SSK).  The  Navy  investigated 
alternative  test  strategies  as  part  of  the  TEMP  update 
process.  The  approved  strategy  asserts  that  the  primary  data 
for  assessing  this  capability  will  come  from  three  sources: 
related  sonar  system  testing  on  Los  Angeles  class  submarines, 
the  use  of  an  onboard  training  system  that  injects  simulated 
acoustic  targets  into  the  combat  system,  and  the  possibility 
for  future  testing  against  real  allied-nations’  SSKs  when 
security  rules  are  relaxed. 

•  The  Block  III  design  will  require  shock  testing  of  the 
Common  Weapons  Launcher  and  the  Virginia  Payload  Tube 
(VPT)  hatch.  The  Navy  plans  to  complete  VPT  hatch  shock 
qualification  testing  in  April  2013  to  support  the  first  Block 
III  delivery  in  August  2014. 

•  The  Navy  is  performing  verification  and  validation  of  the 
Transient  Shock  Analysis  (TSA)  modeling  method  used  for 
the  design  of  Virginia  class  Block  III  items.  The  Navy  plans 
to  accredit  the  TSA  modeling  method  in  March  2013. 

•  The  Navy  plans  to  update  the  Vulnerability  Assessment 
Report  to  include  the  Block  III  modifications  in  July  2014. 


Assessment 

•  The  Navy  achieved  test  efficiencies  by  combining  the 
operational  testing  of  several  programs  into  a  series  of  test 
events.  Since  submarine  platform  and  mission  systems 
testing  are  interdependent,  the  consolidation  of  the  Virginia 
class  testing  with  A-RCI  sonar,  acoustic  arrays,  Mk  48 
torpedoes,  and  the  AN/BYG-1  testing  increased  test  efficiency 
and  enabled  a  more  complete  end-to-end  evaluation  of  the 
Virginia  class  submarine’s  mission  performance.  This  testing 
also  provided  insights  into  the  effectiveness  and  suitability  of 
each  individual  system  and  weapon  the  Navy  uses  on  other 
classes  of  submarines. 

•  The  FOT&E  event  in  the  Arctic  was  adequate;  however,  the 
transfer  and  analysis  of  the  data  were  significantly  delayed 
and  the  Navy  did  not  retain  some  test  data.  DOT&E’s 
assessment  of  Virginia 's  effectiveness  in  the  Arctic 
environment  and  Virginia  s  susceptibility  to  low-frequency 
fixed  passive  sonar  arrays  will  be  contained  in  an  early  FY13 
classified  report. 

•  DOT&E’s  classified  report  on  Virginia  s  modernization 
FOT&E,  issued  in  November  2012,  concluded  the  following: 

-  Virginia  s  operational  effectiveness  is  dependent  on  the 
mission  conducted.  The  modernization  of  the  sonar  and 
fire  control  systems  (A-RCI  and  AN/BYG-1)  with  the 
Advanced  Processor  Build  09  software  did  not  change 
(improve  or  degrade)  the  performance  of  the  Virginia  class 
for  the  missions  tested.  DOT&E’s  assessment  of  mission 
effectiveness  remains  the  same  for  ASW,  ISR,  High 
Contact  Management,  Situational  Awareness,  and  Mine 
Avoidance.  DOT&E’s  overall  assessment  of  Information 
Assurance  remains  unchanged  from  IOT&E,  although  the 
new  software  represents  an  improvement  in  Information 
Assurance  over  previous  systems. 

-  Although  Virginia  was  not  effective  for  some  of  the 
missions  tested,  it  remains  an  effective  replacement  for  the 
Los  Angeles  class  submarine,  providing  similar  mission 
performance  and  improved  covertness. 

-  Testing  to  examine  AS  W- Attack  and  situational  awareness 
in  a  high-surface-ship  density  environment  was  adequate 
for  the  system  software  tested,  but  not  adequate  for  the 
software  version  fielded.  After  completion  of  operational 
testing,  the  Navy  issued  software  changes  intended  to 
address  the  severe  performance  problems  observed  with 
the  Wide  Aperture  Array.  The  Navy  has  not  completed 
operational  testing  on  the  new  software,  which  is  fielded 
on  deployed  submarines.  DOT&E  assesses  that  the  late 
fix  of  the  array’s  deficiencies  is  a  result  of  the  Navy’s 
schedule-driven  development  processes,  which  fields  new 
increments  without  completing  adequate  developmental 
testing. 

-  The  Navy  collected  adequate  data  to  assess  the  suitability 
of  the  sonar  and  fire  control  systems.  Insufficient  data 
were  collected  to  reassess  the  suitability  of  Virginia ’s 
hull,  mechanical,  electrical,  or  electronic  systems; 
however,  these  data  were  not  expected  to  demonstrate 
significantly  different  reliability  compared  to  what  was 
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observed  in  IOT&E.  Of  note,  the  installation  of  the  new 
Advanced  Processor  Build  09  of  the  A-RCI  sonar  system 
on  Virginia  class  submarines  will  degrade  the  reliability  of 
the  sonar  system  on  these  submarines  relative  to  what  was 
demonstrated  in  the  IOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
progress  in  addressing  22  of  the  30  recommendations 
contained  in  the  November  2009  classified  BLRIP  report  and 
the  October  2011  classified  FOT&E  report.  Of  the  outstanding 
recommendations,  the  significant  unclassified  ones  are: 

1.  Test  against  an  SSK  threat  surrogate  in  order  to  evaluate 
Virginia 's  capability,  detectability,  and  survivability  against 
modern  diesel-electric  submarines. 

2.  Conduct  FOT&E  to  examine  Virginia  s  susceptibility  to 
airborne  ASW  threats  such  as  Maritime  Patrol  Aircraft  and 
helicopters. 


•  FY12  Recommendations.  The  Navy  should: 

1.  Coordinate  the  Virginia,  A-RCI,  andAN/BYG-1  TEMPs 
and  utilize  Undersea  Enterprise  Capstone  documents  to 
facilitate  testing  efficiencies. 

2.  Complete  the  verification,  validation,  and  accreditation  of 
the  TSA  method  used  for  Virginia  class  Block  III  items. 

3.  Repeat  the  FOT&E  event  to  determine  Virginia  s 
susceptibility  to  low-frequency  active  sonar  and  Virginia  s 
ability  to  conduct  ASUW  in  a  low-frequency  active 
environment.  This  testing  should  include  a  Los  Angeles 
class  submarine  operating  in  the  same  environment  to 
enable  comparison  with  the  Virginia  class. 

4.  Address  the  21  recommendations  that  are  included  in  the 
November  2012  DOT&E -published  FOT&E  report  on 
Virginia  s  modernization  and  the  associated  sonar  and 
combat  control  systems.  In  particular,  the  Navy  should 
re-evaluate  operational  effectiveness  on  a  submarine  with  a 
repaired  Wide  Aperture  Array. 
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Standard  Missile-6  (SM-6) 


Executive  Summary 

•  The  Navy  completed  the  Phase  2  modeling  and  simulation  in 
support  of  IOT&E  in  July  2012.  Phase  2  was  an  extensive 
modeling  and  simulation  effort  that  examined  Standard 
Missile-6  (SM-6)  battlespace  with  the  legacy  Aegis  Weapon 
System  but  not  the  Navy  Integrated  Fire  Control-Counter  Air 
(NIFC-CA)  or  Aegis  Baseline  9  capability. 

•  The  Navy  will  not  demonstrate  achievement  of  all  of  the  SM-6 
Capability  Production  Document  performance  requirements 
until  the  fielding  of  the  NIFC-CA  From  the  Sea  capability  in 
FY14/15.  The  Navy  plans  to  demonstrate  NIFC-CA  From 
the  Sea  SM-6  capability  during  FOT&E  as  documented  in  the 
SM-6  Test  and  Evaluation  Master  Plan. 

•  In  IOT&E  Phase  1  flight  tests,  SM-6  demonstrated  significant 
new  capabilities  against  maneuvering  targets,  low-altitude 
targets,  and  targets  with  electronic  countermeasures, 
successfully  completing  7  of  12  intercept  attempts.  Within 
the  constraints  of  the  legacy  Aegis  combat  system,  SM-6  also 
demonstrated  the  longest  downrange  engagement  range  for 

a  Standard  Missile  to  date.  IOT&E  Phase  2  modeling  and 
simulation  confirmed  SM-6  performance  demonstrated  in 
flight  test. 

•  To  demonstrate  corrective  actions  to  suitability  anomalies 
discovered  during  IOT&E,  the  Navy  conducted  a  series  of 
high-temperature  wind  tunnel  and  flight  tests.  The  results 
of  that  testing  demonstrated  the  corrective  actions  were 
effective;  however,  the  unexpected  discovery  of  insulation 
inter-layer  delamination  on  three  of  five  wind  tunnel  test 
articles  questions  the  finality  of  the  Navy’s  corrective  actions. 
Additional  testing  is  ongoing. 

•  An  unresolved  performance  anomaly  from  flight-testing 
affects  SM-6  effectiveness.  The  Phase  2  modeling  and 
simulation  testing  confirmed  this.  The  Navy  is  working  to 
develop  corrective  actions  for  this  performance  deficiency; 
however,  testing  of  these  actions  has  not  been  scheduled. 

System 

•  SM-6  is  the  latest  evolution  of  the  Standard  Missile  family  of 
fleet  air  defense  missiles  that  incorporates  components  from 
two  existing  Raytheon  product  lines:  the  SM-2  Block  IV 
and  the  Advanced  Medium-Range  Air-to-Air  Missile 
(AMRAAM). 

•  SM-6  is  employed  from  cruisers  and  destroyers  equipped  with 
Aegis  combat  systems. 


•  The  SM-6  seeker  and  terminal  guidance  electronics  derive 
from  technology  developed  in  the  AMRAAM  program.  SM-6 
retains  the  legacy  Standard  Missile  semi-active  radar  homing 
capability. 

•  SM-6  receives  midcourse  flight  control  from  the  Aegis  combat 
system  via  ship’s  radar;  terminal  flight  control  is  autonomous 
via  the  missile’s  active  seeker  or  supported  by  the  Aegis 
combat  system  via  the  ship’s  illuminator. 

Mission 

•  The  Joint  Force  Commander/Strike  Group  Commander  will 
use  SM-6  for  fleet  air  defense  against  fixed-/rotary-winged 
targets  and  anti-ship  missiles  operating  at  altitudes  ranging 
from  very  high  to  sea-skimming. 

•  The  Joint  Force  Commander  will  use  SM-6  as  part  of  the 
NIFC-CA  concept  to  provide  extended-range,  over-the-horizon 
capability  against  at-sea  and  overland  threats. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  The  Navy  completed  Phase  2  of  the  IOT&E  in  July  2012  in 
accordance  with  the  DOT&E-approved  operational  test  plan. 
Phase  2  was  an  extensive  modeling  and  simulation  effort  that 


examined  legacy  Aegis  Weapon  System  SM-6  battlespace  but 
not  the  NIFC-CA  or  Aegis  Baseline  9  capability. 
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•  The  Navy  conducted  high-temperature  wind  tunnel  tests  to 
verify  correction  of  the  uplink/downlink  antenna  reliability 
deficiency.  The  Navy  plans  to  continue  this  verification  with 
the  conduct  of  follow-on  flight  testing  in  FY13.  The  Navy 
conducted  one  SM-6  flight  test  in  support  of  the  Joint  Land 
Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 
(JLENS)  integration  testing.  Data  from  this  flight  combined 
with  the  FY13  flight  tests  will  provide  some  but  not  all  of  the 
data  needed  to  verify  the  uplink/downlink  antenna  deficiencies 
have  been  corrected. 

Assessment 

•  The  Navy  will  not  demonstrate  all  of  the  SM-6  Capability 
Production  Document  performance  requirements  until  the 
fielding  of  the  NIFC-CA  From  the  Sea  capability  in  FY14/15. 

•  In  Phase  1  IOT&E,  SM-6  demonstrated  significant  new 
capabilities  against  maneuvering  targets,  low-altitude  targets, 
and  targets  with  electronic  countermeasures,  successfully 
completing  7  of  12  intercept  attempts.  The  current  capabilities 
of  the  legacy  Aegis  SPY-1  B/D  and  associated  combat  system 
are  unable  to  demonstrate  the  full  operational  capability  of 
the  SM-6.  Within  those  constraints,  SM-6  demonstrated  the 
longest  downrange  engagement  range  for  a  Standard  Missile 
to-date.  Phase  2  modeling  and  simulation  confirmed  SM-6 
performance  demonstrated  in  flight  test  with  the  legacy  Aegis 
combat  system. 

•  As  an  excursion  during  the  Phase  2  modeling  and  simulation 
activity,  the  Navy  conducted  a  number  of  trials  using  third 
party  sensors  similar  to  the  NIFC-CA  capability.  The  trials 
indicated  that  the  SM-6  battlespace  will  be  significantly 
expanded  once  these  capabilities  are  fielded. 

•  A  performance  deficiency  discovered  during  IOT&E  remains 
unresolved.  The  Phase  2  modeling  and  simulation  trials 
confirmed  this.  The  Navy  is  exploring  corrective  actions; 


however,  implementation  and  testing  of  these  corrective 
actions  are  not  scheduled. 

•  The  high-temperature  wind  tunnel  tests  of  the  uplink/downlink 
antenna  reliability  deficiency  examined  the  antenna  sealant 
material  fixes  and  the  insulation  bonding  manufacturing 
process  improvements.  The  trials  recorded  no  anomalies 
against  these  fixes;  however,  the  unexpected  discovery  of 
insulation  inter-layer  delamination  on  three  of  five  wind  tunnel 
test  articles  questions  the  finality  of  the  Navy’s  corrective 
actions.  Coupled  with  the  data  collected  on  the  JLENS 
integration  flight  test,  the  data  are  insufficient  to  assess 
corrective  action  efficacy  on  the  overall  uplink/downlink 
antenna  reliability  deficiency.  DOT&E  will  continue  to  collect 
data  on  upcoming  SM-6  FOT&E  flight  tests  and  will  re-assess 
effectiveness  and  suitability  when  sufficient  data  are  available. 

•  Based  upon  combined  data  from  the  IOT&E  and 
developmental/operational  flight  tests,  the  SM-6  does  not 
meet  the  flight  reliability  criteria  established  by  USD(AT&L) 
for  full-rate  production.  DOT&E  will  continue  to  collect 
reliability  data  during  upcoming  SM-6  FOT&E  firings  and  will 
re-assess  suitability  at  the  conclusion  of  these  tests. 

•  First  seen  in  developmental  testing,  the  Mk  54  Safe- Arm 
Device  anomaly  carried  forward  into  IOT&E  with  additional 
occurrences.  The  Phase  2  modeling  and  simulation  trials 
confirmed  that  the  sensitivity  of  missile  lethality  is  dependant 
on  the  fuze  mode,  target,  and  engagement  conditions. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  addressing 
the  previous  recommendations. 

•  FY12  Recommendation. 

1.  Until  reliability  deficiencies  are  resolved,  the  Navy  should 
consider  issuing  tactics  that  employ  multiple  missiles  for 
certain  targets. 
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Surveillance  Towed  Array  Sensor  System  (SURTASS) 
and  Compact  Low  Frequency  Active  (CLFA)  System 


Executive  Summary 

•  The  Navy  has  installed  one  engineering  developmental 
model  and  two  production  Compact  Low  Frequency  Active 
(CLFA)  systems  on  three  of  the  five  Western  Pacific-based 
tactical  auxiliary  general  ocean  surveillance  (T-AGOS)  ships. 
Installation  of  the  CLFA  system  on  remaining  T-AGOS  ships 
is  not  planned. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  commenced  IOT&E  on  the  Surveillance  Towed 
Array  Sensor  System  (SURTASS)/CLFA  during  fleet  exercise 
Valiant  Shield  12  in  September  2012.  IOT&E  will  complete 
in  FY13.  The  analysis  of  test  data  collected  is  still  in  progress. 
No  preliminary  evaluation  is  available. 

System 

•  SURTASS/CLFA  is  a  low  frequency,  passive  and  active, 
acoustic  surveillance  system  installed  on  T-AGOS  ships  as  a 
component  of  the  Integrated  Undersea  Surveillance  System 
(IUSS). 

•  SURTASS  provides  passive  detection  of  quiet  nuclear 
and  diesel  submarines  and  enables  real-time  reporting  of 
surveillance  information  to  Anti-Submarine  Warfare  (ASW) 
commanders. 

•  CLFA  is  a  low  frequency,  active  sonar  system  developed  to 
provide  an  active  detection  capability  of  quiet  submarines 
operating  in  environments  that  support  active  sonar 
propagation. 

•  The  system  consists  of: 

-  A  T-AGOS  host  ship  with  array-handling  equipment 

-  A  towed  vertical  string  of  active  acoustic  projectors 

-  A  towed  horizontal  twin  line  (TL-29A)  acoustic  array 

-  An  integrated  common  processor  (ICP)  for  processing 
active  and  passive  acoustic  data 

-  A  communications  segment  to  provide  connectivity  to 
shore -based  IUSS  processing  facilities  and  to  fleet  ASW 
commanders 

Mission 

•  Maritime  Component  Commanders  employ  T-AGOS  ships 
equipped  with  SURTASS/CLFA  systems  to  provide  active 
and  passive  acoustic  sensors  for  long-range  ASW  detection, 
classification,  and  tracking  of  submarines  in  support  of  theater 
naval  operations. 

•  Maritime  Component  Commanders  use  SURTASS/CLFA  to 
protect  naval  ships  from  threat  submarines  and  to  provide 


accurate  targeting  information  to  theater  ASW  forces  to 
prosecute  the  threat  submarines. 

Major  Contractors 

•  Overall  Integrator:  Maritime  Surveillance  Systems  Program 
Office  (PMS  485) 

•  ICP:  Lockheed  Martin  -  Manassas,  Virginia 

•  CLFA  Projectors:  BAE  -  Nashua,  New  Hampshire 

•  CLFA  Handling  System:  Naval  Facilities  Engineering  Service 
Center  (NAVFAC  ESC)  (Government  Lab)  -  Port  Hueneme, 
California 

•  High  Frequency  Marine  Mammal  Monitoring  Sonar: 

Scientific  Solutions  Incorporated  (SSI)  -  Nashua,  New 
Hampshire 

•  TL-29A  Towed  Arrays:  Lockheed  Martin  -  Syracuse,  New 
York 
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Activity 

•  After  the  August  2010  DOT&E  Operational  Assessment 
of  SURTASS/CLFA  installed  on  USNS  Able ,  the  Navy 
determined  that  additional  system  development  was  required 
to  address  system  reliability,  automated  detection,  and  active 
tracking  concerns  before  the  system  could  proceed  to  IOT&E. 

-  In  January  2011,  the  Navy  conducted  at-sea  test 
(AST)  11-2  to  assess  developmental  changes  to  improve 
reliability. 

-  In  October  2011,  the  Navy  conducted  AST  11-4  to  assess 
developmental  changes  to  improve  automated  detection 
and  active  tracking  performance. 

-  In  May  2012,  the  Navy  conducted  AST  12-lAas  an 
advanced  engineering  analysis  and  system  validation  to 
inform  a  system  certification  decision. 

-  On  August  29,  20 12,  the  Program  Executive  Office, 
Submarines  certified  the  CLFA  system  ready  to  proceed  to 
IOT&E. 

•  DOT&E  approved  the  IOT&E  test  plan  for  SURTASS/CLFA 
on  September  4,  2012,  and  agreed  to  use  operationally 
realistic  test  and  exercise  data  collected  during  a  scheduled 
fleet  exercise,  Valiant  Shield  12,  which  commenced  on 
September  10,  2012,  in  the  Western  Pacific,  as  well  as  a 
dedicated  test  phase  not  associated  with  Valiant  Shield  12. 

•  In  September  2012,  COTF  and  DOT&E  commenced  IOT&E 
on  the  SURTASS/CLFA  system  installed  on  USNS  Effective 
(T-AGOS-21).  Testing  focused  on  SURTASS/CLFA 
contribution  to  coordinated  ASW  against  threat  diesel  and 
nuclear  submarines  and  included  both  passive  and  active  sonar 
from  multiple  air  and  sea  platforms.  IOT&E  will  complete  in 
FY13. 

•  The  Navy  acquired  and  installed  one  engineering 
developmental  model  and  two  production  CLFA  systems 
on  three  of  the  five  Western  Pacific-based  T-AGOS  ships. 
Installation  of  the  CLFA  system  on  remaining  T-AGOS  ships 
is  not  planned. 


Assessment 

•  The  conduct  of  IOT&E  during  an  operationally  relevant  fleet 
exercise,  Valiant  Shield  12,  allowed  data  collection  that  will 
provide  insight  into  the  effectiveness  of  SURTASS/CLFA 
as  a  primary  contributor  to  theater  ASW.  The  data  will 
allow  assessment  of  the  ASW  commander’s  ability  to  utilize 
SURTASS/CLFA  contact  reports  with  other  ASW  assets  to 
protect  surface  ships  and  prosecute  threat  submarines. 

•  Due  to  a  limitation  in  submarine  support  availability, 
substantially  less  data  were  obtained  to  support  evaluation 
of  the  long-range  active  detection  capability  against 
threat-representative  submarines  than  was  planned.  Additional 
testing  is  required  to  allow  an  adequate  assessment. 

•  The  analysis  of  test  data  collected  during  the  combined 
SURTASS/CLFA  IOT&E  and  Valiant  Shield  12  fleet  exercise 
is  still  in  progress.  No  preliminary  evaluation  is  available. 
DOT&E  expects  to  issue  a  formal  test  report  in  FY13  after 
completion  of  IOT&E.  The  Navy  has  executed  testing 
completed  thus  far  in  accordance  with  the  DOT&E-approved 
test  plan. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  and  Program 
Office  are  satisfactorily  addressing  previous  recommendations. 
The  Navy  implemented  the  FY11  recommendation  to  conduct 
IOT&E  during  a  fleet  exercise.  The  Program  Office  continued 
system  development  consistent  with  FY 1 1  recommendations. 
Correction  of  deficiencies  identified  in  COTF’s  and  DOT&E ’s 
operational  assessment  reports  will  be  validated  during 
IOT&E. 

•  FY12  Recommendation. 

1.  The  Navy  should  complete  remaining  IOT&E  to  include  a 
follow-on  test  event  that  allows  an  adequate  determination 
of  long-range  active  detection  capability  against 
threat-representative  submarines. 
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T-AKE  Lewis  &  Clark  Class  of  Auxiliary  Dry  Cargo  Ships 


Executive  Summary 

•  The  second  FOT&E  test  series  (OT-11IB)  for  the  T-AKE  Lewis 
&  Clark  program  specifically  addressed  the  ship’s  Chemical, 
Biological,  and  Radiological  (CBR)  defenses;  Information 
Assurance  (IA);  and  mine  susceptibility  including  magnetic 
silencing  (degaussing). 

•  While  the  majority  of  deficiencies  were  verified  as  corrected, 
a  major  deficiency  involving  corrosion  in  the  Countermeasure 
Water  Wash  Down  (CMWWD)  piping  system  still  remains. 

System 

•  T-AKE  Lewis  &  Clark  is  a  class  of  non-combatant  ships 
operated  by  the  Military  Sealift  Command  (MSC)  designed 
to  carry  dry  cargo,  ammunition,  and  fuel  (in  limited  amounts) 
for  naval  combat  forces  at  sea.  There  are  14  ships  in  the  class; 
11  ships  are  under  contract  for  the  Combat  Logistics  Force 
and  3  additional  ships  for  the  Maritime  Prepositioning  Force 
(Future). 

•  The  T-AKE  is: 

-  Constructed  to  commercial  standards  (American  Bureau 
of  Shipping)  with  some  additional  features  to  increase  its 
survivability  in  hostile  environments  such  as  the  Advanced 
Degaussing  System  to  reduce  the  ship’s  magnetic  signature 
against  mines,  shock  resistance  in  selected  equipment, 
and  increased  damage  control  measures  in  firefighting  and 
stability 

-  Operated  by  civilian  mariners  from  the  MSC  and  a  small 
Navy  military  detachment 

-  Propelled  with  a  single  shaft  and  propeller;  driven  by 
electric  motors  powered  by  diesel  generators 

Mission 

The  Maritime  Component  Commander  can  employ  the  T-AKE 

Lewis  &  Clark  class  of  ships  to: 


•  Re-supply  other  ships  while  connected  underway  using 
Standard  Tensioned  Replenishment  Alongside  Method  rigs 
and  embarked  helicopters 

•  Move  cargo  and  ammunition  between  a  port  and  a  larger 
consolidating  replenishment  ship,  which  stays  with  the 
Carrier/Expeditionary  Strike  Group 

•  Be  part  of  the  hybrid  combination  of  ships  of  the  Maritime 
Prepositioning  Force  (Future) 

Major  Contractor 

General  Dynamics  National  Steel  and  Shipbuilding 

Company  -  San  Diego,  California 


Activity 

•  The  Navy  conducted  OT-I11B  February  through  May  2012  in 
accordance  with  the  DOT&E -approved  test  plan.  It  focused 
on  final  resolution  of  1A  and  survivability  vulnerabilities. 

The  tests  were  intended  to  continue  evaluation  of  T-AKE 
operational  effectiveness  and  suitability,  verify  deficiencies 
identified  in  IOT&E  and  an  earlier  FOT&E  were  corrected, 
and  complete  deferred  OT&E. 

•  The  Navy  installed  a  new  intruder  detection  system  on 
T-AKE  12  and  integrated  the  navigation  and  engineering 
control  system  with  the  ship’s  network. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force’s  (COTF)  Blue  Team  completed  an  Operational 


Information  Assurance  Vulnerability  Evaluation  on  T-AKE  12 
while  in-port  in  February  2012.  COTF’s  Red  Team  completed 
a  penetration  test  while  underway  during  April  and  May  2012. 

•  During  the  May  at-sea  period,  COTF  completed  CBR  and 
magnetic  mine  survivability  tests.  The  Navy  installed  a  new 
Improved  Point  Detection  System-Lifecycle  Replacement 
(IPDS-LR)  on  T-AKE  12,  which  was  evaluated  during  the 
CBR  test. 

•  The  Surface  Warfare  Center  Panama  City  Division  conducted 
the  Advanced  Mine  Simulation  System  test  series  during 
May  2012. 
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Assessment 

•  During  the  Operational  Information  Assurance  Vulnerability 
Evaluation,  COTF’s  Blue  Team  found  a  number  of  potential 
Category  1  vulnerabilities  within  the  various  components  of 
T-AKE  12.  During  the  penetration  test,  however,  none  of  the 
exploits  the  Red  Team  ran  against  T-AKE  12  were  able  to 
penetrate  the  ship’s  premise  (outer  most)  router.  However, 
the  Red  Team  did  achieve  unauthorized  access  to  the  ship’s 
computing  system  during  the  penetration  test,  which  ultimately 
allowed  the  team  to  escalate  user  privileges  and  gain  system 
access. 

•  During  the  CBR  test,  the  ship’s  crew  was  able  to  don  the 
necessary  personal  and  collective  equipment  to  defend 
themselves  against  the  effects  of  CBR  attacks  and  hazards. 

The  newly  installed  1PDS-LR  functioned  properly  and  watch 
officers  were  able  to  respond  to  audible  and  visual  alarms 
on  the  bridge.  Unlike  U.S.  Navy  ships,  however,  on  MSC 
ships  the  IPDS-LR  does  not  automatically  activate  the  ship’s 
general  or  chemical  alarm.  The  crew  must  manually  activate 


this  alarm,  followed  by  a  general  announcement  that  there  is  a 
CBR  emergency. 

•  The  CMWWD  system  was  adequate.  However,  COTF 
conducted  an  additional  test  on  T-AKE  1  (5  years  old)  that 
revealed  that  the  system  was  severely  degraded  due  to 
corrosion  of  the  mild  carbon  steel  piping. 

•  The  analysis  of  test  data  collected  while  T-AKE  operated  near 
the  Advanced  Mine  Simulation  System  (AMISS)  is  ongoing 
and  no  preliminary  evaluation  is  available.  DOT&E  expects  to 
issue  a  formal  test  report  in  2QFY13. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
addressed  all  previous  recommendations. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Resolve  IA  Category  I  vulnerabilities. 

2.  Resolve  the  CMWWD  system’s  corrosion  problem. 
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Tomahawk  Missile  and  Weapon  System 


Executive  Summary 

•  As  demonstrated  during  FY12  test  flights,  the  Tomahawk 
Weapon  System  continues  to  meet  Navy  standards  for 
reliability  and  performance. 

•  As  assessed  in  the  February  2012  FOT&E  report  to  Congress, 
the  Tomahawk  Weapon  System  continues  to  be  effective  and 
suitable. 

•  DOT&E  considers  the  planned  Operational  Test  Launch 
program  to  be  adequate  for  continued  verification  of  system 
reliability  and  accuracy.  DOT&E  places  high  value  on 
continuing  flight  data  collection  to  evaluate  end-to-end  system 
performance  and  reliability  for  all  deployed  and  deployable 
Tomahawk  missile  variants.  However,  in  FY12,  the  Navy 
discontinued  flight  testing  of  the  fielded  Block  111  missiles, 
which  are  to  remain  in  operational  use  until  FY20. 

System 

•  The  Tomahawk  Land  Attack  Missile  is  a  long-range,  land 
attack  cruise  missile  designed  for  launch  from  submarines  and 
surface  ships. 

•  Production  of  Tomahawk  Block  II  and  III  missiles  is 
complete.  There  are  four  fielded  variants:  a  Block  II  with  a 
nuclear  warhead  (not  deployed),  a  Block  III  with  a  unitary 
conventional  warhead,  a  Block  III  with  a  conventional 
submunitions  warhead,  and  a  Block  IV  with  a  conventional 
unitary  warhead. 

•  Block  IV  Tomahawk  is  in  production  as  the  follow-on  to 
the  Block  III  conventional  unitary  warhead  variant.  These 
missiles  are  produced  at  lower  cost  and  provide  added 
capability,  including  the  ability  to  communicate  with 
command  and  control  and  be  redirected  to  an  alternate  target 
during  flight. 

•  The  Tomahawk  Weapon  System  also  includes  the  Tomahawk 
Command  and  Control  System  (TC2S)  and  the  shipboard 


Tactical  Tomahawk  Weapon  Control  Systems  (TTWCS). 

The  TC2S  and  TTWCS  provide  for  command  and  control, 
targeting,  mission  planning,  distribution  of  Tomahawk  tactical 
and  strike  data,  and  post-launch  control  of  Block  IV  missiles. 

Mission 

The  Joint  Force  Commander  employs  the  Tomahawk  Weapon 

System  for  long-range,  precision  strikes  against  land  targets. 

Major  Contractor 

•  Missile  element:  Raytheon  Missile  Systems  -  Tucson, 
Arizona 

•  Weapon  Control  System  element:  Lockheed  Martin  -  Valley 
Forge,  Pennsylvania 

•  Command  and  Control  element:  QinetiQ  North  America 
LLC  -  San  Jose,  California,  and  Boeing  Inc.  -  St.  Louis, 
Missouri 


Activity 

•  DOT&E  submitted  the  FOT&E  operational  test  report  to 
Congress  in  February  2012. 

•  In  accordance  with  the  DOT&E-approved  Test  and 
Evaluation  Master  Plan  and  operational  test  plan,  the  Navy 
continued  to  conduct  FOT&E  Operational  Test  Launches 
to  verify  reliability  and  performance  of  fielded  Block  III 
and  IV  Tomahawk  missiles,  their  associated  weapon  control 
systems,  and  the  TC2S.  The  Navy  conducted  a  total  of  eight 
Tomahawk  missile  test  launches  in  FY12. 

•  In  FY12,  DOT&E  conducted  a  comparative  flight  reliability 
analysis  of  over  200  operational  Tomahawk  firings  conducted 


by  the  fleet  during  FY 1 1  to  the  reliability  demonstrated  during 
all  Operational  Test  Launches  conducted  to  date. 

Assessment 

•  As  demonstrated  during  FY12  test  flights,  the  Tomahawk 
Weapon  System  continues  to  meet  Navy  standards  for 
reliability  and  performance.  As  reported  to  Congress  in 
the  FY12  FOT&E  report,  the  Tomahawk  Weapon  System 
continues  to  be  effective  and  suitable. 

•  DOT&E  considers  the  current  Operational  Test  Launch 
program  for  all  Tomahawk  missile  variants  to  be  adequate 
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for  continued  verification  of  system  reliability  and  accuracy. 
However,  while  Block  IV  testing  is  funded  through  FY13, 
the  Navy  is  not  funding  further  Block  III  test  launches.  The 
Block  III  missiles  are  to  remain  in  operational  use  until  FY20. 
DOT&E  places  high  value  on  continuing  flight  test  data  to 
evaluate  end-to-end  system  performance  and  reliability  for  all 
deployed  and  deployable  Tomahawk  missile  variants. 

•  The  DOT&E  analysis  of  FY 1 1  operational  Tomahawk  firings 
concluded  that  Tomahawk  fleet  firing  reliability  is  consistent 
with  observed  FOT&E  Operational  Test  Launch  results. 


Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
addressed  all  previous  recommendations. 

•  FY12  Recommendation. 

1.  The  Navy  should  resource  the  FOT&E  Operational  Test 
Launch  series  to  include  testing  of  all  fielded  missile 
variants. 
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Vertical  Take-Off  and  Landing  Unmanned  Aerial  Vehicle 

(VTUAV) 


Executive  Summary 

•  The  Navy  stopped  production  of  the  MQ-8B  air  vehicle  after 
procuring  23  MQ-8Bs.  They  have  not  conducted  IOT&E  on 
the  MQ-8B  air  vehicles.  The  Navy  is  considering  replacement 
of  the  Schweizer  333  (MQ-8B)  airframe  with  the  Bell  407 
(MQ-8C).  MQ-8C  development  is  in  response  to  a  U.S. 
Special  Operations  Command  Joint  Universal  Operational 
Needs  Statement. 

•  The  Test  and  Evaluation  Master  Plan  (TEMP)  approved 
in  2007  is  outdated  and  does  not  contain  a  clear  path  to 
successful  development,  integration,  and  testing  of  the  Vertical 
Take-Off  and  Landing  Unmanned  Aerial  Vehicle  (VTUAV). 

•  The  Navy  deployed  MQ-8B  Fire  Scout  systems  aboard 
Navy  frigates  USS  Simpson  and  USS  Klakring  during  2012. 
The  USS  Simpson  supported  forward  presence  and  training 
operations  off  the  west  coast  of  Africa  while  the  USS  Klakring 
conducted  operations  off  the  Horn  of  Africa.  The  Simpson 
and  Klakring  deployments  aboard  Navy  frigates  demonstrated 
that  the  VTUAV  has  potential  to  provide  the  commander  with 
valuable  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 
once  the  program  addresses  shortcomings  identified  during 
developmental  testing.  Demonstrated  component  lifecycle 
reliability  is  well  below  program  planning  levels. 

•  Past  and  present  MQ-8B  deployments  aboard  frigates  and 
operations  in  support  of  the  ISR  Task  Force  resulted  in  a 
critical  shortage  of  spare  parts. 

•  Operational  deployments  and  developmental  testing  confirmed 
that  system  reliability,  availability,  communications  relay,  and 
documentation  remain  unsatisfactory. 

•  Moving  from  the  MQ-8B  to  the  MQ-8C  airframe  may  or  may 
not  improve  VTUAV  reliability. 

System 

•  The  VTUAV  is  a  helicopter-based  tactical  Unmanned  Aerial 
System  comprised  of  up  to  three  Fire  Scout  air  vehicles  with 
payloads,  a  shipboard  integrated  Ground  Control  Station 
with  associated  Tactical  Common  Data  Link  (TCDL),  and 
the  Unmanned  Aerial  Vehicle  Common  Automatic  Recovery 
System  (UCARS). 


•  The  Navy  intends  the  VTUAV  with  the  MQ-8B  to  have  the 
following  capabilities: 

-  Combat  radius  -  1 10  nautical  miles 

-  Endurance  at  combat  radius  -  3  hours  on  station 

-  Target  Identification  -  Small  fast  attack  boats  at 
6  kilometer  range 

-  Initial  payload  consists  of  the  AN/AAQ-22D  Bright  Star  II 
electro-optical  and  infrared  imaging  system  with  laser 
designator 

•  The  Navy  is  considering  replacement  of  the  Schweizer  333 
(MQ-8B)  airframe  with  the  Bell  407  (MQ-8C). 

Mission 

Aviation  detachments  equipped  with  VTUAVs  perform 
reconnaissance,  surveillance,  target  acquisition,  and 
communications  relay  missions  in  support  of  littoral  Anti 
Submarine  Warfare,  Anti-Surface  Warfare,  and  Mine  Warfare 
operations.  System  deployments  during  2012  provided 
reconnaissance  and  surveillance  to  units  conducting  combat 
operations  ashore. 

Major  Contractor 

Northrop  Grumman-Ryan  Aeronautical  -  San  Diego,  California 


Activity 

•  The  Navy  has  stopped  production  of  the  MQ-8B  air  vehicle 
after  procuring  23  MQ-8Bs.  Instead,  the  Navy  has  focused 
on  development  of  the  MQ-8C  air  vehicle  (also  known  as  the 
“Endurance  Upgrade”)  as  a  Rapid  Deployment  Capability. 


The  Navy  is  considering  replacement  of  the  Schweizer  333 
(MQ-8B)  airframe  with  the  Bell  407  (MQ-8C). 

•  MQ-8C  development  is  in  response  to  a  U.S.  Special 
Operations  Command  Joint  Universal  Operational  Needs 


VTUAV  219 


NAVY  PROGRAMS 


Statement.  The  Program  Office  issued  a  sole  source  contract 
to  Northrop  Grumman  for  MQ-8C  development. 

•  Northrop  Grumman  is  conducting  MQ-8C  risk  reduction  flight 
testing  using  internal  research  and  development  funds. 

•  The  Navy  is  considering  procurement  of  3 1  MQ-8C 
air  vehicles  (28  production  air  vehicles  plus  3  research, 
development,  and  engineering  air  vehicles). 

•  The  Navy  has  not  conducted  IOT&E  on  the  23  MQ-8B  air 
vehicles  already  procured. 

•  The  Navy  is  working  on  the  integration  of  weapons  and  a 
search  radar  capability  to  the  MQ-8B  air  vehicle  in  response  to 
a  Navy  Urgent  Operational  Need. 

•  The  Navy  deployed  MQ-8B  Fire  Scout  systems  aboard 
Navy  frigates  USS  Simpson  and  USS  Klakring  during  2012. 
The  USS  Simpson  supported  forward  presence  and  training 
operations  off  the  west  coast  of  Africa  while  the  USS  Klah-ing 
conducted  operations  off  the  Horn  of  Africa.  The  Navy’s 
Commander,  Operational  Test  and  Evaluation  Force  published 
a  Quick  Reaction  Assessment  in  September  2012. 

•  One  Fire  Scout  system  continues  to  support  1SR  Task  Force 
operations  in  Afghanistan. 

•  The  Navy  lost  two  MQ-8B  aircraft  in  April  2012.  Operators 
aboard  USS  Simpson  intentionally  ditched  one  air  vehicle 
after  encountering  problems  with  the  recovery  system.  The 
Navy  recovered  this  air  vehicle.  The  second  air  vehicle  was 
destroyed  after  crashing  while  operating  in  support  of  the  1SR 
Task  Force  in  Afghanistan. 

•  In  May  2012,  Naval  Air  Systems  Command  published  an 
Interim  Summary  Report  addressing  the  status  of  MQ-8B 
developmental  testing. 

•  The  Navy  plans  to  update  the  TEMP  by  April  2013;  this 
update  is  expected  to  expand  the  scope  of  IOT&E. 

Assessment 

•  The  TEMP  approved  in  2007  is  outdated  and  does  not  contain 
a  clear  path  to  successful  completion  of  IOT&E.  The  TEMP 
does  not  clearly  define  the  objectives  of  near-term  testing, 
nor  does  it  prioritize  future  upgrades  such  as  search  radar  and 
weapons  integration. 

•  Developmental  testing  during  2012  verified  the  correction 
of  several  deficiencies  that  adversely  affected  system 
performance.  Software  updates  corrected  the  target  location 
error  and  payload  automatic  caging  deficiencies.  The  software 
now  allows  skilled  operators  to  conduct  dual  air  vehicle 
operations. 

•  The  Simpson  and  Klab'ing  deployments  demonstrated  that 
the  VTUAV  has  the  potential  to  provide  the  commander 
with  valuable  ISR  once  the  program  addresses  shortcomings 
identified  during  developmental  testing. 

•  Data  collected  during  operational  deployments  and 
developmental  testing  show  that  the  VTUAV  system  has 
performed  to  a  satisfactory  level  in  the  areas  of  air  vehicle 
operations,  maintainability,  compatibility,  interoperability, 
human  factors,  and  safety. 


•  Demonstrated  component  lifecycle  reliability  is  well  below 
program  planning  levels.  This  resulted  in  unacceptable 
values  for  Availability,  Mean  Flight  Hours  Between 
Operational  Mission  Failures,  and  Mean  Flight  Hours  Between 
Unscheduled  Maintenance  Actions,  preventing  the  program 
from  entering  into  IOT&E.  This  poor  reliability  adversely 
affects  performance  of  the  forward-deployed  systems  and 
increases  the  workload  of  the  aviation  detachments. 

•  The  Navy  based  Fire  Scout  spare  parts  budgeting  on  design 
reliability  and  operating  tempo  in  support  of  the  Littoral 
Combat  Ship.  Past  and  present  deployments  aboard  frigates 
and  operations  in  support  of  the  ISR  Task  Force  combined 
with  significantly  lower  lifecycle  reliability  have  caused  a 
critical  shortage  of  spare  parts. 

•  Data  collected  during  operational  deployments  and 
developmental  testing  confirmed  that  system  performance  in 
the  areas  of  reliability,  availability,  communications  relay,  and 
documentation  remain  unsatisfactory. 

•  The  Navy  has  yet  to  assess  several  critical  areas  related 
to  VTUAV  performance.  These  include  tactics,  logistics 
supportability,  training,  and  manning.  Each  of  these  areas,  in 
and  of  themselves,  could  render  the  system  not  effective  or  not 
suitable  during  IOT&E. 

•  Moving  from  the  MQ-8B  to  the  MQ-8C  airframe  may  or  may 
not  improve  VTUAV  reliability.  While  the  Navy  will  not  see 
some  failure  modes  specific  to  the  MQ-8B  on  the  MQ-8C,  the 
MQ-8C  includes  systems  not  found  on  the  MQ-8B. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has 
made  satisfactory  progress  on  three  of  the  four  FY 1 1 
recommendations.  It  has  developed  overland  ISR  standard 
operating  procedures  and  an  operator  training  syllabus 

for  aviation  detachments  operating  VTUAV.  The  Navy 
has  also  established  an  office  to  review  and  coordinate  all 
Navy  UAS  development  and  fielding,  which  addresses  the 
recommendation  to  conduct  an  end-to-end  review  of  its 
command  and  control  network  to  facilitate  the  dissemination 
of  near-real-time  video.  Given  the  delay  in  IOT&E,  the  one 
remaining  recommendation  to  expand  the  scope  of  IOT&E 
will  be  addressed  as  the  Navy  updates  the  TEMP. 

•  FY12  Recommendations.  The  Navy  should: 

1 .  Conduct  in-depth  shipboard  testing  to  fully  characterize 
TCDL  performance  to  include  air  vehicle  orientation  in 
relation  to  the  ship  and  ship  orientation  in  relation  to  the  air 
vehicle. 

2.  Use  available  data  to  conduct  a  formal  assessment  of 
VTUAV  tactics,  logistics  supportability,  training,  and 
manning  to  identify  areas  of  risk  to  successful  IOT&E. 

3.  Conduct  a  failure  mode  analysis  between  the  MQ-8C  and 
the  MQ-8B  to  determine  which  failure  modes  are  common, 
which  failures  modes  do  not  transfer  from  the  MQ-8B  to  the 
MQ-8C,  and  which  failure  modes  are  unique  to  the  MQ-8C. 
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AIM-120  Advanced  Medium-Range  Air-to-Air  Missile 

(AMRAAM) 


Executive  Summary 

•  In  2009,  key  stakeholders,  including  the  Program  Office 
and  DOT&E,  suspended  progression  of  the  AIM-120D  to 
operational  testing  due  to  four  performance  and  reliability 
deficiencies.  Raytheon  addressed  the  four  deficiencies,  and 
DOT&E  signed  the  revised  AIM-120D  Test  and  Evaluation 
Master  Plan  (TEMP)  and  separate,  detailed  operational  test 
plan  on  May  25,  2012. 

•  The  Air  Force  completed  an  Operational  Test  Readiness 
Review  on  May  31,  2012,  and  certified  the  AIM-120D 
Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM)  to 
begin  operational  testing  in  June  2012.  AIM-120D  operational 
testing  will  consist  of  seven  live  missile  shots  and  multiple 
captive-carry  events.  The  Services  are  projected  to  complete 
operational  testing  in  FY14. 

•  During  operational  testing  to  date,  the  Air  Force  has 
accomplished  three  AIM-120D  shots,  the  first  of  which  was 
unsuccessful.  The  unsuccessful  shot  was  due  to  a  reliability 
failure  of  a  legacy  Shortened  Control  Actuation  System. 

The  Air  Force  has  re-executed  this  shot  and  is  awaiting  data 
analysis  to  determine  its  success. 

•  The  AMRAAM  Electronic  Protection  Improvement  Program 
(EPIP)  is  a  software  upgrade  to  AIM-120C3-C7  variants 
currently  in  integrated  testing,  under  the  separate  EPIP  TEMP 
that  DOT&E  approved  in  April  2012. 

System 

•  The  AIM- 120  AMRAAM  is  an  all-weather,  radar-guided  air 
to-air  missile  with  capability  in  both  the  beyond-visual-range 
and  within-visual-range  arenas.  A  single  launch  aircraft  can 
engage  multiple  targets  with  multiple  missiles  simultaneously 
when  using  AMRAAM. 

•  The  AMRAAM  program  develops  and  incorporates  phased 
upgrades  periodically.  AMRAAM  EPIP  is  a  software  upgrade 
to  AIM-120C3-C7. 


•  The  latest  version,  the  AIM-120C7,  incorporated  an  upgraded 
antenna,  receiver,  signal  processor,  and  new  software 
algorithms  to  counter  new  threats.  The  use  of  smaller  system 
components  created  room  for  future  growth. 

•  The  AIM-120D  is  currently  in  development,  and  the  Air  Force 
intends  for  it  to  deliver  perfonnance  improvements  beyond  the 
AIM-120C7  through  the  use  of  an  internal  GPS,  an  enhanced 
datalink,  and  new  software. 

Mission 

•  The  Air  Force  and  Navy,  as  well  as  several  foreign  military 
forces,  use  various  versions  of  the  AIM- 120  AMRAAM  to 
shoot  down  enemy  aircraft. 

•  All  U.S.  fighter  aircraft  use  the  AMRAAM  as  the  primary 
beyond-visual-range  air-to-air  weapon. 

Major  Contractor 

•  Raytheon  Missile  Systems  -  Tucson,  Arizona 

•  Rocket  Motor  Subcontractors: 

-  Alliant  Techsystems  (ATK)  -  Arlington,  Virginia 

-  Nammo  -  Raufoss,  Norway 


Activity 

AIM-1 2  0D 

•  Production  of  the  AIM-120D  began  in  2006,  and 
developmental  testing,  which  included  three  integrated 
developmental/operational  test  missile  shots,  began  in 
2007. 

•  In  2009,  key  stakeholders,  including  the  Program  Office 
and  DOT&E,  suspended  progression  of  the  AIM-120D  to 
operational  testing  due  to  four  performance  and  reliability 
deficiencies,  including  missile  lockup,  built-in  test  failures, 


aircraft  integration  problems,  and  poor  GPS  satellite 
acquisition. 

•  The  Air  Force  accomplished  the  final 

developmental/operational  test  shot  in  August  2011. 
Raytheon  addressed  the  four  deficiencies,  and  DOT&E 
signed  the  TEMP  and  separate  operational  test  plan  on 
May  25,  2012.  The  Air  Force  conducted  an  Operational 
Test  Readiness  Review  in  May  2012,  and  certified  the 
program  to  begin  operational  testing  in  June  2012.  Testing 
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is  progressing  in  accordance  with  the  DOT&E-approved 
TEMP  and  test  plan. 

•  AIM-120D  operational  testing  will  consist  of  seven  live 
missile  shots  and  multiple  captive-carry  events.  The 
Services  are  projected  to  complete  operational  testing  in 
FY14. 

AMRAAM  EPIP 

•  DOT &E  approved  the  AMRAAM  EPIP  TEMP  on 
April  13,  2012,  after  which  integrated  testing  began. 

Lot  Acceptance  Test/Rocket  Motors 

•  Beginning  in  December  2011,  propellant  hot  spots,  burn 
through,  and  failures  at  low  temperatures  (-65  degrees 
Fahrenheit)  caused  unpredictable  Lot  Acceptance  Test 
(LAT)  failures.  ATK,  the  subcontractor  who  produces  the 
rocket  motors,  continues  to  investigate  these  failures. 

•  The  Program  Office,  Raytheon,  and  AMRAAM  safety 
communities  coordinated  to  certify  Nammo  to  become  an 
approved  alternative  rocket  motor  supplier. 

•  The  Program  Office  has  suspended  performance-based 
payments  until  resolution  of  the  shortage  of  rocket  motors 
due  to  unacceptable  LAT  performance. 

Assessment 

•  The  Air  Force  originally  planned  for  AIM-  120D  to  begin 
operational  testing  in  2008;  it  is  now  approximately  four  years 
behind  schedule. 

•  The  Air  Force  adequately  addressed  fixes  to  the  four 
performance  and  reliability  deficiencies  that  precluded 
AIM-120D  from  proceeding  to  operational  testing: 

-  Raytheon  fixed  the  built-in  test  false  alarm  with  a  software 
modification. 

-  Raytheon  fixed  the  GPS  failure  problem  with  a  GPS  filter, 
which  was  verified  with  multiple  laboratory  events  and  a 
successful  live  missile  shot. 

-  The  Air  Force  and  Navy  requirements  sponsors  clarified 
software  requirements,  and  subsequent  software  changes 
reduced  the  frequency  of  missile  lockups. 

-  Software  changes  mitigated  some  aircraft  integration 
problems,  but  more  problems  will  need  resolution  in  a 


System  Improvement  Program  following  operational 
testing. 

•  Since  the  start  of  operational  testing,  the  Air  Force  has 
executed  three  live  missile  shots.  The  first  was  a  reliability 
failure  most  likely  due  to  a  grounding  wire  common  to  all 
AMRAAM  variants  going  back  to  the  AIM-120C5.  The 
second  live  missile  shot  was  the  first  AIM- 120D  launch  from 
an  F-22  and  resulted  in  a  direct  hit  on  the  drone.  The  third  was 
a  re-execution  of  the  first  live  missile  shot  and  the  Air  Force  is 
awaiting  data  analysis  to  determine  its  success.  Additionally, 
captive-carry  performance  has  been  nominal  and  the  Air  Force 
has  yet  to  quantify  system  reliability. 

•  The  shortage  in  rocket  motors  due  to  unacceptable  LAT 
performance  should  not  significantly  affect  AIM- 120D  testing, 
but  it  has  created  a  backlog  in  production.  After  Insensitive 
Munitions  certification  in  October  2012,  Nammo  will  be  the 
only  AIM-120D  and  AIM-120C7  rocket  motor  producer  for 
the  foreseeable  future.  The  government  and  Raytheon  are  still 
reviewing  a  path  forward  for  ATK  production. 

Recommendations 

•  Status  of  Previous  Recommendations.  TheFY07 
recommendation  for  the  Program  Office  to  seek  changes  to  the 
Air  Force’s  full-scale  and  sub-scale  target  programs  to  ensure 
proper  target  presentation,  target  reliability,  and  availability 
remains  valid.  The  Air  Force  has  adequately  addressed  all 
other  previous  recommendations. 

•  FY12  Recommendations. 

1.  The  Air  Force  should  complete  a  System  Improvement 
Program  within  one  year  of  the  completion  of  operational 
testing  to  address  any  remaining  aircraft  integration 
problems,  as  well  as  any  problems  discovered  during 
operational  testing. 

2.  The  Program  Office  should  continue  root  cause  analysis  of 
rocket  motor  LAT  failures  and  keep  the  Air  Force,  Navy, 
and  OSD  updated  on  potential  impacts  to  cost,  schedule, 
and  performance. 
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Air  Operations  Center  -  Weapon  System  (AOC-WS) 


Executive  Summary 

•  The  Air  Operations  Center  -  Weapon  System  (AOC-WS)  10.1 
is  a  system-of-systems  that  contains  numerous  third-party 
software  applications,  including  the  Global  Command  and 
Control  System  -  Joint  (GCCS-J),  Theater  Battle  Management 
Core  Systems  -  Force  Level  (TBMCS-FL),  Master  Air 
Attack  Plan  Toolkit  (MAAPTK),  and  Joint  Automated  Deep 
Operations  Coordination  System  (JADOCS). 

•  The  Air  Force  tests  AOC-WS  10.1  software  upgrades  during 
Recurring  Events  (REs)  and  refers  to  each  software  upgrade 
by  the  event  during  which  it  was  tested.  For  example, 
AOC-WS  10.1  RE  11  is  the  software  upgrade  that  was  tested 
during  RE11. 

•  The  Air  Force  conducted  operational  testing  of  RE  11  in 
March  2012  using  older  servers  due  to  a  short  timeline  to 
integrate  newer  servers  for  test. 

•  The  605th  Test  and  Evaluation  Squadron  (TES)  conducted  a 
regression  test  in  May  2012  on  newer,  more  powerful  servers. 
This  test  included  additional  software  fixes  to  GCCS-J 
provided  by  the  Defense  Information  Systems  Agency  (DISA). 

•  From  July  through  September  2012,  the  Air  Force  conducted 
a  third  phase  of  RE11  operational  testing  at  the  613th  AOC, 
Hickam  AFB,  Hawaii. 

•  Using  RE  11,  operators  can  successfully  execute  all  critical 
missions  and  produce  threshold  or  larger-sized  target  lists  and 
Air  Tasking  Orders  on  schedule.  The  Common  Operational 
Picture  exceeded  the  AOC  requirement  for  updating  and 
displaying  the  threshold  number  of  tracks  in  near  real-time. 

•  RE  1 1  demonstrated  interoperability  with  other  mission-critical 
systems.  Operational  testing  showed  that  RE  11  addresses 

the  significant  legacy  RE10  and  GCCS-J  deficiencies 
identified  from  previous  testing.  RE  11  provides  a  significant 
improvement  to  AOCs  both  in  internal  functionality,  as  well  as 
in  interoperability  with  Combatant  Commands. 

•  The  605th  TES  is  still  evaluating  RE  1 1  ’s  operational 
suitability;  however,  data  collected  to  date  indicate  the 
need  for  some  improvement.  RE  11  requires  significant 
manpower  to  maintain  the  security  posture  of  the  system. 

Each  operational  AOC  has  site-unique  requirements  that  each 
unit  must  incorporate  into  RE11.  The  AOC-WS  Help  Desk 
is  working  to  adapt  their  enabling  concept  to  include  the 
enhanced  capabilities  in  RE11. 

•  Users  find  training  acceptable,  and  the  overall  system 
availability  is  acceptable.  Information  Assurance  risk  is 
acceptable,  and  the  system  has  obtained  an  Authority  to 
Operate. 

System 

•  The  AOC-WS  is  the  senior  command  and  control  element 
of  the  U.S.  Air  Force’s  Theater  Air  Control  System  and 


provides  operational-level  command  and  control  of  air,  space, 
and  cyberspace  operations,  as  well  as  joint  and  combined 
air,  space,  and  cyberspace  operations.  Capabilities  include 
command  and  control  of  joint  theater  air  and  missile  defense; 
time-sensitive  targeting;  and  intelligence,  surveillance,  and 
reconnaissance  management. 

•  The  AOC-WS  10.1  (AN/USQ-163  Falconer) 
is  a  system-of-systems  that  contains  numerous 
third-party-developed  software  applications  and  commercial 
off-the-shelf  products.  Each  third-party  system  integrated  into 
the  AOC-WS  provides  its  own  programmatic  documentation. 

•  The  AOC-WS  consists  of: 

-  Commercial  off-the-shelf  hardware 

-  Separate  third-party  software  applications  (GCCS-J, 
TBMCS-FL,  MAAPTK,  and  JADOCS),  from  which  the 
AOC-WS  draws  its  capabilities 

-  Additional  third-party  systems  that  accept,  process, 
correlate,  and  fuse  command  and  control  data  from 
multiple  sources  and  share  them  through  multiple 
communications  systems 

•  AOC-WS  10.1  operates  on  several  different  local  area 
networks  (LANs),  including  Secret  Internet  Protocol  Router 
Network,  Joint  Worldwide  Intelligence  Communications 
System,  and  a  coalition  LAN,  when  required.  The  LANs 
connect  the  core  operating  system  and  primary  applications  to 
joint  and  coalition  partners  supporting  the  applicable  area  of 
operation.  Users  can  assess  web-based  applications  through 
the  Defense  Information  Systems  Network. 

•  The  Air  Force  tests  AOC-WS  10.1  software  upgrades  during 
REs.  The  Air  Force  refers  to  each  software  upgrade  by  the 
event  during  which  it  was  tested.  For  example,  AOC-WS  10.1 
RE  11  is  the  software  upgrade  that  was  tested  during  RE11. 

•  The  future  AOC-WS  10.2  will  be  the  first  increment  for 
modernization. 
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Mission 

The  Commander,  Air  Force  Forces,  or  the  Joint/Combined  Forces 
Air  Component  Commander  use  the  AOC-WS  to  exercise  control 
of  joint  (or  combined)  air  forces  including  planning,  directing, 
and  assessing  air,  space,  and  cyberspace  operations  to  meet 
operational  objectives  and  guidance. 


Major  Contractors 

•  AOC-WS  10.1  Production  Center:  Jacobs  Technology 
Inc.,  Engineering  and  Technology  Acquisition  Support 
Services  -  Hampton,  Virginia 

•  AOC-WS  10.2  Modernization:  Northrop  Grumman 
Corporation  -  Hampton,  Virginia 


Activity 

•  The  Air  Force  has  developed  an  RE  test  cycle  for  major 
AOC-WS  10.1  upgrades,  along  with  lower-level  testing  events 
to  sustain  interoperability  and  Information  Assurance,  and  to 
provide  low-risk  upgrades  to  third-party  systems  as  required. 
DOT&E  approved  an  update  to  the  program’s  Test  and 
Evaluation  Master  Plan  in  December  2011,  and  approved  an 
overarching  operational  test  plan  for  all  AOC-WS  10.1  testing 
in  February  2012. 

•  The  Air  Force  conducted  developmental  testing  of  RE  1 1 
in  January  2012  and  made  several  fixes  to  mission-critical 
problems.  The  Air  Force  conducted  a  subsequent  regression 
test  in  February  2012. 

•  RE1 1  introduced  GCCS-J  Integrated  Imagery  and  Intelligence 
as  the  intelligence  and  targeting  capability  provider  for 
AOC-WS  and  upgraded  TBMCS-FL  with  Maintenance 
Release  2  to  migrate  to  current  Modernized  Integrated 
DataBase  (MIDB)  2.1.  Without  the  TBMCS-FL  and  GCCS-J 
upgrades,  the  AOC-WS  is  restricted  to  using  a  less-capable 
MIDB  2.0. 

•  Air  Combat  Command  (ACC)  conducted  a  risk  analysis  of 
the  11  open  Category  I  test  problem  reports  following  the 
developmental  and  regression  test.  ACC  determined  the 
best  course  of  action  to  meet  current  operational  need  was  to 
proceed  to  operational  testing  and  fielding  of  RE11. 

•  ACC  developed  a  plan  of  action  and  milestones  to  address  the 
1 1  open  Category  I  deficiencies  at  the  time  of  operational  test 
initiation.  This  decision  was  to  be  revisited  if  any  additional 
Category  I  problems  were  discovered  during  operational  test. 

•  The  Air  Force  conducted  operational  testing  of  RE1 1  in 
March  2012.  The  AOC-WS  Program  Office  had  addressed 
4  of  the  1 1  open  Category  I  problem  reports  by  the  start  of 
the  operational  test.  The  GCCS-J  portion  of  this  testing  was 
conducted  on  older  servers;  previous  testing  has  shown  that 
these  servers  have  insufficient  computing  power.  The  newer, 
more  powerful  servers  were  not  available  in  time  to  meet  the 
initial  integration  and  test  schedule. 

•  The  605th  Test  and  Evaluation  Squadron  conducted  regression 
testing  of  RE  11  in  May  2012  using  newer,  more  powerful 
servers  to  host  the  GCCS-J  software.  DISA  had  also  provided 
several  fixes  to  GCCS-J  in  the  form  of  an  Update  1  software 
release. 

•  The  Air  Force  conducted  a  third  phase  of  operational  testing  of 
RE11  at  the  613th  AOC,  Hickam  AFB,  Hawaii,  July  through 
September  2012.  This  testing  focused  on  the  ability  of  the 
install  team  to  correctly  upgrade  and  configure  the  AOC  from 


legacy  RE  10  to  RE  11,  and  perform  backup  and  recovery 
actions  on  RE11. 

•  DISA  provided  additional  fixes  to  GCCS-J  to  the  Air  Force 
for  incorporation  into  the  next  upgrade  (RE  12),  currently 
scheduled  for  developmental  and  operational  testing  in 
December  2012. 

•  The  Air  Force  conducted  all  RE1 1  testing  in  accordance  with 
the  DOT&E-approved  test  plan. 

Assessment 

•  The  Air  Force  adequately  tested  RE1 1  through  a  combination 
of  developmental  and  operational  testing. 

•  RE  11  can  successfully  execute  all  critical  missions  and 
produce  threshold  or  larger-sized  target  lists  and  Air  Tasking 
Orders  on  schedule.  The  Common  Operational  Picture 
was  able  to  exceed  the  AOC  requirement  for  updating  and 
displaying  the  threshold  number  of  tracks  in  near  real-time. 

•  RE  1 1  demonstrated  interoperability  with  other  mission-critical 
systems.  Operational  testing  showed  that  RE1 1  could  address 
the  significant  AOC-WS  and  GCCS-J  deficiencies  identified 
from  previous  testing. 

•  Users  successfully  demonstrated  the  ability  to  transfer  Joint 
Forces  Air  Component  Command  responsibilities.  Users 
encountered  three  Category  I  critical  deficiencies  with 
third-party  software  systems  that  are  used  within  RE11,  but 
these  problems  did  not  prevent  mission  accomplishment. 

ACC  accepted  these  limitations  while  encouraging  prompt 
resolution  of  these  problems.  RE11  provides  a  significant 
improvement  to  AOCs,  in  both  internal  functionality  as  well 
as  their  ability  to  interoperate  with  their  respective  Combatant 
Commands. 

•  The  605th  TES  is  still  evaluating  RE  1  l’s  operational 
suitability;  however,  data  collected  to  date  indicate  the  need  for 
some  improvement.  RE1 1  requires  significant  manpower  to 
maintain  the  security  posture  of  the  system  on  a  recurring  basis 
and  the  AOC-WS  password  change  procedures  are  complex 
and  inordinately  lengthy.  The  GCCS-J  backup  and  recovery, 
and  server  password  change  procedures  for  the  newer  servers 
are  deficient.  System  administrators  require  extensive  training 
in  order  to  competently  manage  the  system.  Each  operational 
AOC  has  site-unique  requirements  that  must  be  incorporated 
into  RE11.  The  AOC-WS  Help  Desk  is  worldng  to  adapt 
their  enabling  concept  to  include  the  enhanced  capabilities  in 
RE11.  Users  find  training  acceptable  and  the  overall  system 
availability  meets  requirements.  Information  Assurance  risk 
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is  acceptable,  and  the  system  has  obtained  an  Authority  to 
Operate. 

•  The  key  to  successful  testing  and  fielding  of  RE  1 1  has  been 
closer  collaboration  between  the  AOC-WS  Program  Office  and 
DISA  to  ensure  GCCS-J  meets  the  operational  needs  of  the 
AOCs.  As  a  result  of  AOC-WS  tester  involvement  in  GCCS-J 
testing,  the  AOC-WS  testers  identified  critical  problems  early 
for  corrective  action. 

•  The  high-priority  fixes  DISA  provided  to  GCCS-J  in  Update  1 
improved  data  exchange  between  GCCS-J  and  RE1 1. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  is 
adequately  addressing  previous  recommendations.  Two  of  the 
F  Y 1 1  recommendations  are  long-term  improvements  that  are 
currently  being  worked  and  still  need  continued  effort. 

1 .  Coordinate  with  third-party  programs  to  ensure  that  critical 
AOC-WS  third-party  systems  (such  as  GCCS-J)  have 
testable  requirements  that  meet  AOC-WS  requirements. 
Requirements  should  be  properly  vetted  within  the 
appropriate  user  and  program  communities  for  schedule  and 
funding  priority. 


2.  Ensure  the  AOC-WS  users  and  test  community  continue 
to  actively  participate  in  GCCS-J  developmental  and 
operational  testing  and  collaborate  to  develop  a  capability  to 
adequately  test  GCCS-J  to  AOC-WS  threshold  stress  levels. 

FY12  Recommendations. 

1.  The  Air  Force  should  address  suitability  concerns  with 
RE  11,  especially  enabling  system  managers  to  more  easily 
maintain  the  security  posture  of  the  RE11. 

2.  Per  the  recommendation  from  DOT&E’s  Information 
Assurance/Interoperability  assessment  memorandum  to 
the  Director  of  Operations  (J3),  Joint  Staff,  the  Joint  Staff 
should  establish  or  identify  a  systems  integration  group 
for  command  and  control  systems.  These  command 

and  control  systems  will  be  responsible  for  providing 
comprehensive  oversight/management  of  joint  command 
and  control  systems  and  mission-critical  interfaces,  with 
particular  emphasis  on  joint  data  fusion  and  operations 
centers,  such  as  the  AOC.  Systems  that  should  be 
addressed  include,  but  are  not  limited  to,  GCCS-J,  MIDB, 
TBMCS-FL,  JADOCS,  Joint  Targeting  Toolbox,  and 
AOC-WS. 
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ALR-69A  Radar  Warning  Receiver  (RWR) 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  accomplished  IOT&E  from  May  18  through 
July  16,  2012.  The  flight  tests  were  flown  at  the  Fallon  Range 
Training  Complex,  Nevada,  and  the  Multi-Spectral  Test  and 
Training  Environment  at  Eglin  AFB,  Florida,  resulting  in  a 
total  threat  exposure  time  of  12  hours.  The  system  logged  a 
total  of  204  hours  of  operating  time  during  IOT&E. 

•  DOT&E  assessed  the  system  as  not  operationally  effective 
but  operationally  suitable.  The  system  was  not  operationally 
effective  because  it  did  not  consistently  provide  the  aircrew 
timely  and  accurate  threat  information  and  the  system 
demonstrated  a  random  threat  symbol  splitting  deficiency. 
Threat  symbol  splitting  occurs  when  one  threat  signal  received 
by  the  system  produces  multiple  threat  symbols  at  different 
azimuths  on  the  cockpit  display.  This  degrades  the  aircrew’s 
situational  awareness  as  to  which  displayed  threats  are 
“real”  and  where  those  real  threats  are  located  and  inhibits 

the  aircrew’s  ability  to  appropriately  react  to  the  threat(s) 
in  a  timely  manner.  The  details  of  the  DOT&E  assessment 
are  presented  in  DOT&E’s  classified  IOT&E  report,  dated 
October  2012. 

•  Although  the  Air  Force  System  Program  Office  (SPO)  and 
Raytheon  conducted  hardware-in-the-loop  (FIWIL)  tests  to 
demonstrate  the  threat  signal  splitting  deficiency  has  been 
resolved,  DOT&E  does  not  think  HWIL  testing  by  itself  is 
adequate  to  verify  the  deficiency  has  been  resolved  and  that 
the  software  update  did  not  induce  any  other  adverse  system 
performance. 

System 

•  The  ALR-69A  is  a  Radar  Warning  Receiver  (RWR)  that 
detects,  identifies,  and  locates  threat  electronic  signals. 

•  The  Core  ALR-69A  RWR  is  designed  to  improve  performance 
over  the  Air  Force’s  primary  RWR  system,  the  ALR-69,  by 
enhancing: 

-  Detection  range  and  time 

-  Accuracy  of  threat  identification 

-  Location  of  threat  emitter  systems 

-  Performance  in  a  dense  signal  environment 

-  Reliability  and  maintainability 


1  -  Legacy  ALR-69  Components  3  -  Radar  Receiver 

2  -  Primary  ALR-69  Components  4  -  Countermeasure  Signal  Processor 


•  The  system  integrates  with  transport  and  fighter  aircraft.  The 
lead  platform  is  the  C-130F1,  with  other  platforms  possibly  to 
be  added  at  a  later  date. 

•  Core  ALR-69A  RWR  components  include: 

-  Radar  receivers  (previously  the  digital  quadrant  receivers) 

-  Modular  Countermeasures  Signal  Processor  (previously 
the  countermeasures  computer) 

-  Control  indicator 

-  Azimuth  indicator 

Mission 

•  Combatant  Commanders  will  use  ALR-69A  to  enhance  the 
survivability  of  transport,  fighter,  and  Special  Operations 
aircraft  on  missions  that  penetrate  hostile  areas. 

•  Aircrews  use  the  ALR-69A  to  provide  indication  of  ground 
and  airborne  radar  threats  in  order  to  support  threat  avoidance 
maneuvers  and/or  timely  use  of  defensive  countermeasures. 

Major  Contractor 

Raytheon,  Space  and  Airborne  Systems  -  Goleta,  California 


Activity 

•  The  Air  Combat  Command  413th  Flight  Test  Squadron  issued 
a  Developmental  Test  and  Evaluation  (DT&E)  report  in 
February  2012  to  the  SPO  at  Warner  Robins  Air  Force  Base, 
Georgia.  The  report  covered  testing  accomplished  between 


February  2010  and  May  2011.  The  SPO  briefed  DT&E  results 
to  the  Milestone  Decision  Authority  in  November  2011. 

•  DOT&E  approved  the  AFOTEC  IOT&E  Test  Concept 
on  February  12,  2012,  and  approved  the  IOT&E  test  plan 
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and  revised  Test  and  Evaluation  Master  Plan  (TEMP)  on 
February  16,  2012. 

•  AFOTEC  began  suitability  flights  in  April  2012,  and  held 
IOT&E  test  crew  training  in  early  May  2012,  with  several 
suitability  flight  tests  occurring  shortly  afterwards. 

•  AFOTEC  began  operational  flight  testing  on  May  18,  2012, 
at  the  Fallon  Range  Training  Complex,  Nevada.  As  a  result 
of  weather  and  competing  range  priorities,  the  system  was 
only  exposed  to  range  threats  for  approximately  4  hours. 

In  response,  AFOTEC  flew  three  additional  missions  on 
June  12,  13,  and  15  at  the  Multi-Spectral  Test  and  Training 
Environment  at  Eglin  AFB,  Florida.  The  combination  resulted 
in  a  total  threat  exposure  time  of  12  hours.  The  system  logged 
a  total  of  204  hours  of  operating  time  during  IOT&E. 

•  AFOTEC  conducted  the  final  test  of  IOT&E  on  July  16,  2012. 
It  was  a  ground-based  test  designed  to  evaluate  the  system’s 
reprogramming  capability. 

•  On  August  1,  2012,  the  Air  Force  SPO  and  Raytheon 
conducted  a  HW1L  test  comparing  the  performance  of  the 
ALR-69A  software  that  was  used  during  IOT&E  to  updated 
software  generated  to  correct  a  threat  symbol  splitting 
deficiency  observed  throughout  IOT&E. 

•  AFOTEC  did  not  execute  testing  in  accordance  with  the 
DOT&E-approved  test  plan.  AFOTEC  deviated  from  the 
test  plan  in  the  following  areas:  AFOTEC  pre -briefed 
aircrews  about  the  type  and  location  of  threats,  which  reduced 
DOT&E’s  ability  to  determine  the  contribution  the  ALR-69A 
made  to  the  aircrew’s  situational  awareness;  several  missions 
lacked  operational  realism;  and  aircrew  questionnaires  did  not 
incorporate  inputs  from  DOT&E  to  improve  the  quality  of  the 
data  generated  from  the  questionnaires. 

Assessment 

•  DOT&E  assessed  the  ALR-69A  system  as  not  operationally 
effective  but  operationally  suitable.  The  system  was  not 
operationally  effective  because  it  did  not  consistently  provide 


the  aircrew  timely  and  accurate  threat  information  and 
the  system  demonstrated  a  random  threat  symbol  splitting 
deficiency.  Threat  symbol  splitting  occurs  when  one  threat 
signal  received  by  the  system  produces  multiple  threat  symbols 
at  different  azimuths  on  the  cockpit  display.  This  degrades  the 
aircrew’s  situational  awareness  as  to  which  displayed  threats 
are  “real”  and  where  those  real  threats  are  located  and  inhibits 
the  aircrew’s  ability  to  appropriately  react  to  the  threat(s) 
in  a  timely  manner.  The  details  of  the  DOT&E  assessment 
are  presented  in  DOT&E’s  classified  IOT&E  report,  dated 
October  2012. 

•  Although  the  SPO  and  Raytheon  conducted  HW1L  tests  to 
demonstrate  the  threat  signal  splitting  deficiency  has  been 
resolved,  F1W1L  testing  by  itself  is  not  adequate  to  verify  the 
deficiency  has  been  resolved  and  that  the  software  update  did 
not  induce  any  other  adverse  system  performance. 

Recommendations 

•  Status  of  Previous  Recommendations.  DOT&E  last  reported 
on  this  program  in  FY09.  The  Air  Force  has  satisfactorily 
addressed  previous  recommendations. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Correct  aircrew  threat  audio  warnings  so  that  the 
appropriate  tone  is  associated  with  the  correct  status  of  the 
threat. 

2.  Improve  the  timeliness  and  accuracy  of  threat  information 
provided  to  the  aircrew  to  improve  the  aircrew’s  situational 
awareness. 

3.  Conduct  flight  testing  to  verify  the  system  is  operationally 
effective  and  that  the  software  upgrade  implemented  by 
the  SPO  and  Raytheon  corrects  the  threat  symbol  splitting 
deficiency  and  did  not  degrade  system  performance  in  any 
other  area. 
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B-2  Extremely  High  Frequency  (EHF) 
Satellite  Communications  (SATCOM) 
and  Computer  Upgrade  Program 


Executive  Summary 

•  Developmental  flight  testing  that  began  in  September  2010 
completed  in  April  2012. 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  IOT&E  from  June  through  August  2012 
to  assess  the  program’s  operational  effectiveness,  suitability, 
and  mission  capability.  The  IOT&E  included  6  sorties  and 

56  hours  of  dedicated  flight  testing  including  navigation,  long 
duration  missions,  and  both  conventional  and  nuclear  weapons 
delivery  events.  IOT&E  further  assessed  effectiveness  and 
suitability  data  across  283  hours  of  developmentaFintegrated 
flight  test  missions  conducted  during  formal  developmental 
testing. 

•  IOT&E  results  indicate  the  system  is  operationally  effective. 
However,  the  limited  flight  test  hours  accumulated  preclude 
DOT&E  from  making  a  definitive  assessment  of  the  system’s 
ability  to  meet  the  legacy  system  demonstrated  Mean  Time 
Between  Critical  Failure  (MTBCF)  of  671  hours.  Therefore, 
continued  monitoring  of  installed  system  performance  in 
operational  aircraft  is  necessary  to  confirm  reliability. 

•  The  Air  Force  plans  to  conduct  the  Full-Rate  Production 
Decision  in  December  2012. 

System 

•  The  B-2  is  a  multi-role,  low-observable  bomber,  capable  of 
delivering  conventional  and  nuclear  munitions.  It  has  four 
turbofan  engines  and  twin  side-by-side  weapons  bays. 

•  B-2  system  avionics  include  a  multi-mode  radar,  GPS-aided 
navigation,  and  a  Defensive  Management  System  for  radar 
warning  functions. 

•  The  B-2  Extremely  High  Frequency  (EHF)  Satellite 
Communications  (SATCOM)  and  Computer  Upgrade  program 
is  designed  to  deliver  capability  across  distinct  increments. 
Increment  1  upgrades  the  core  flight  management  processing 
capability  of  the  B-2  and  lays  the  foundation  for  subsequent 
avionics  upgrades.  Increment  1  replaces  the  existing  aircraft 
flight  management  computers  with  two  new  Integrated 
Processing  Units  and  two  new  Data  Drive  Units  to  increase 
data  storage.  Increment  1  also  re-hosts  the  aircraft  Flight 
Management  Operational  Flight  Program  from  its  legacy  flight 
management  software  programming  language,  JOVIAL,  to  C. 


•  Follow-on  B-2  EHF  SATCOM  Increments  will  remove  the 
legacy  B-2  MILSTAR  AN/ASC-36  Ultra  High  Frequency 
(UHF)/Air  Force  SATCOM  System,  and  add  the  Family 
of  Advanced  Beyond  Line-of-Sight  Terminals  and  a  low 
observable  antenna  to  support  EHF  and  Advanced  EHF 
communications  connectivity.  The  final  Increment  is  planned 
to  be  software-centric  and  provide  full  software  integration  of 
the  B-2  EHF  SATCOM  upgrade,  including  Global  Information 
Grid  connectivity. 

Mission 

•  Combatant  Commanders  use  the  B-2  aircraft  to  attack  global 
targets  during  the  day  or  at  night,  in  all  weather,  in  highly 
defended  threat  areas  at  the  strategic,  operational,  and  tactical 
levels  of  warfare. 

•  Commanders  use  the  B-2  to  engage  high-value,  heavily 
defended  target  sets  including:  command  and  control 
facilities,  airfields,  industrial  complexes,  logistical  and  air 
defense  systems,  lines  of  communication,  and  battlefield 
forces  and  equipment. 

Major  Contractor 

Northrop  Grumman  -  Falls  Church,  Virginia 


Activity 

•  The  Air  Force  conducted  B-2  EHF  SATCOM  Increment  1  •  Developmental  flight  testing  that  began  in  September  2010 

testing  in  accordance  with  the  DOT&E-approved  Test  and  completed  in  April  2012.  AFOTEC  conducted  IOT&E  from 

Evaluation  Master  Plan  and  IOT&E  plan. 
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June  through  August  2012  to  assess  the  program’s  operational 
effectiveness,  suitability,  and  mission  capability.  IOT&E 
included  6  sorties  and  56  hours  of  dedicated  flight  testing 
including  navigation,  long-duration  missions,  and  both 
conventional  and  nuclear  weapons  delivery  events.  IOT&E 
further  assessed  effectiveness  and  suitability  data  across 
283  hours  of  developmental/integrated  flight  test  missions 
conducted  during  formal  developmental  testing. 

•  The  Air  Force  plans  to  conduct  the  Full-Rate  Production 
Decision  in  December  2012. 

Assessment 

•  Results  from  the  IOT&E  indicate  B-2  met  legacy  SATCOM, 
nuclear  and  conventional  weapons  accuracy,  and  navigational 
system  accuracy. 

•  Accumulated  system  flight  test  hours  were  sufficient  to 
evaluate  overall  system  maintainability  consistent  with  the 
B-2  Global  Strike  Concept  of  Operations.  Flowever,  the 


limited  flight  test  hours  accumulated  preclude  DOT&E  from 
making  a  definitive  assessment  of  the  system’s  ability  to 
meet  the  legacy  system  demonstrated  operational  MTBCF 
of  671  hours.  Therefore,  continued  monitoring  of  installed 
system  performance  in  operational  aircraft  is  necessary  to 
confirm  reliability. 

•  Operational  availability  of  96.6  percent  did  not  meet  the  legacy 
system  performance,  but  the  difference  is  not  operationally 
significant. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has 
addressed  all  previous  recommendations. 

•  FY12  Recommendation. 

1 .  The  Air  Force  should  monitor  fielded  performance  to  ensure 
that  system  MTBCF  and  operational  availability  meet  user 
requirements  within  a  narrower  set  of  confidence  bounds  as 
fleet  flying  hours  accumulate. 
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Battle  Control  System  -  Fixed  (BCS-F) 


Executive  Summary 

•  Air  Combat  Command  completed  a  Force  Development 
Evaluation  (FDE)  on  the  Battle  Control  System  -  Fixed 
(BCS-F)  Increment  3,  Release  3.1.3  (R3.1.3)  and  fielded  it  at 
all  U.S.  air  defense  sites  in  December  2011. 

•  Results  from  testing  found  R3 . 1 .3  supports  N orth  American 
Aerospace  Defense  Command  (NORAD)  air  defense 
operations  with  shortfalls  in  training  and  technical  system 
documentation,  system  security  management,  and  system 
combat  identification  operations.  R3.1.3  testing  also  revealed 
significant  deficiencies  in  Information  Assurance  (IA). 

•  The  Air  Force  completed  IOT&E  and  FDE  on  the  BCS-F  R3.2 
at  all  U.S.  air  defense  sites  in  September  2012.  R3.2  presented 
some  IA  improvements  and  achieved  an  interim  Authority  to 
Operate. 

System 

•  The  BCS-F  is  a  tactical  air  battle  management  command 
and  control  system  that  provides  the  two  continental  U.S. 
NORAD  air  defense  sectors,  as  well  as  the  Hawaii  and  Alaska 
Regional  Air  Operation  Centers,  with  commercial  off-the-shelf 
hardware  using  an  open-architecture  software  configuration. 
The  system  operates  within  the  NORAD  air  defense 
architecture  and  is  employed  by  the  U.S.  and  Canada. 

•  The  R3.2  upgrade  includes  the  following  system 
enhancements: 

-  Improved  tactical  datalinks  with  additional  Link  16  and 
Link  1 1  message  types  that  enable  the  operators  to  better 
digitally  control  fighters,  send  amplifying  intelligence 
information,  and  create  a  more  comprehensive  air  picture 

-  Air  Tasking  Order  (ATO)  and  Airspace  Control  Order 
(ACO)  integration  with  theater  battle  management  core 
system  data  sources  that  enables  the  operators  to  view  the 
most  current  ATO/ ACO  and  correlate  the  information  with 
military  aircraft 

-  Adaptation  data  modification  tools  to  enable  system 
administrators  easier  field  changes  to  system  adaptation 
files  and  to  perform  error  checks  with  greater  fidelity 

-  System  control  manager  interface  improvements  that 
enable  the  system  administrator  improved  system 
performance  monitoring  and  diagnostics 

-  Global  Area  Reference  System  coordinate  conversion  tool 
that  facilitates  NORAD  interface  with  global  search  and 
rescue  efforts  ensuring  a  common  set  of  coordinates 


-  Remote  Gateway  Manager  control  through  the  virtual 
network  computing  interface  that  provides  the  operators 
a  complete  picture  of  the  available  datalinks  and  the 
flexibility  to  access  link  information  from  an  operator 
workstation 

-  Auxiliary  server  for  offline  training  and  support 
capabilities  at  the  U.S.  air  defense  sectors 

-  Improved  system  capacities  from  10,300  to  15,000  system 
tracks  to  support  single  sector  continental  United  States 
operations 

Mission 

•  NORAD  and  U.S.  Pacific  Command  Commanders  use  BCS-F 
to  execute  command  and  control  and  air  battle  management  in 
support  of  air  sovereignty  and  air  defense  missions  for  North 
American  Homeland  Defense. 

•  Air  defense  operators  employ  BCS-F  to  conduct  surveillance, 
identification,  and  control  of  U.S.  sovereign  airspace  and 
control  air  defense  assets,  including  fighters,  to  intercept  and 
identify  potential  air  threats  to  U.S.  airspace. 

Major  Contractor 

Thales-Raytheon  Systems  -  Brea,  California 


Activity 

•  Air  Combat  Command  completed  an  FDE  of  R3. 1.3  at  the  Operations  Center,  and  the  Hawaii  Regional  Air  Operations 

System  Support  Facility  at  Tyndall  AFB,  Western  Air  Defense  Center  in  December  2011. 

Sector,  Eastern  Air  Defense  Sector,  Alaska  Regional  Air 
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•  The  Air  Force  fielded  R3.1.3  to  all  U.S.  air  defense  sectors  and 
Canada  in  December  2011. 

•  In  September  2011,  the  Air  Force’s  92nd  Information 
Operations  Squadron  (10  S)  completed  an  insider  threat 
assessment  on  R3.1.3. 

•  The  Air  Force  conducted  developmental  testing  on  R3.2  at  the 
System  Support  Facility  at  Tyndall  AFB  from  July  through 
October  2011  and  January  2012.  Additionally,  the  Air  Force 
accomplished  an  1A  certification  test  at  this  time.  The  Air 
Force  also  conducted  developmental  testing  at  the  U.S. 
operational  sectors  from  February  through  July  2012. 

•  The  Air  Force  conducted  system  interoperability  tests 
in  November  and  December  2011  to  evaluate  datalink 
compliance  with  military  standards. 

•  The  Joint  Interoperability  Test  Command  is  evaluating  critical 
information  exchanges  in  R3.2;  testing  started  in  April  and  will 
end  in  November  2012. 

•  In  April  2012,  the  Air  Force’s  92nd  IOS  conducted  penetration 
and  vulnerability  testing  on  R3.2  at  the  System  Support 
Facility. 

•  The  Air  Force’s  92nd  IOS  conducted  an  insider  threat 
assessment  on  R3.2  at  the  Eastern  Air  Defense  Sector  in 
August  2012. 

•  From  April  through  August  2012,  the  Air  Force  Operational 
Test  and  Evaluation  Center  (AFOTEC)  conducted  IOT&E  on 
R3.2  at  the  System  Support  Facility  and  two  U.S.  operational 
sectors.  Air  Combat  Command  conducted  FDE  at  the  other 
two  operational  sectors. 

•  Air  Combat  Command  began  integrated  developmental  and 
operational  testing  of  R3.2.1  at  the  System  Support  Facility  in 
September  2012.  The  Air  Force  intends  for  R3.2.1  to  provide  a 
fix  for  a  critical  deficiency  discovered  during  R3.2  operational 
testing.  This  deficiency  causes  a  loss  of  situational  awareness 
for  the  operators  conducting  surveillance  of  the  National 
Capital  Region  and  results  in  an  inaccurate  air  picture.  The  Air 
Force  designed  R3.2.1  to  provide  the  ability  to  operate  with 
mandatory  International  Civil  Aviation  Organization  flight  plan 
changes  scheduled  for  November  2012,  and  provide  an  update 
to  local  radar  site  information. 

•  AFOTEC  and  Air  Combat  Command  conducted  operational 
testing  in  accordance  with  a  DOT&E-approved  Test  and 
Evaluation  Master  Plan. 

Assessment 

•  DOT&E  analyses  ofR3.1.3  concluded: 

-  R3.1.3  is  operationally  effective  and  supports  NORAD 
air  defense  operations,  providing  the  ability  to  adequately 
perform  core  competencies  and  tasks  required  to 
accomplish  the  air  defense  mission. 

-  R3.1.3  is  operationally  suitable  with  major  shortfalls 
in  logistics  supportability,  system  documentation,  and 
training.  While  training  on  datalink  operations  and  combat 
identification  have  improved,  system  security  training  and 
training  for  differences  in  each  new  build  still  have  major 
deficiencies. 


-  Shortfalls  in  system  security  management  and  deficiencies 
in  all  1A  assessment  areas  jeopardize  secure  system 
operations.  Test  results  from  the  September  2011  insider 
threat  assessment  indicate  R3. 1.3  is  deficient  in  the  1A 
areas  of  protect,  detect,  react,  and  restore.  The  large 
number  of  outstanding  deficiencies  indicates  the  system’s 
continued  vulnerability  to  detect,  withstand,  or  recover 
from  attacks  by  cyber  threats. 

-  R3. 1.3  demonstrated  adequate  reliability,  maintainability, 
and  availability.  R3.1.3  has  an  average  system  availability 
of  99.98  percent,  with  over  1,118  hours  of  system  operation 
during  operational  test. 

-  Deficiencies  still  exist  in  R3.1  training  for  the  intrusion 
detection  system,  the  firewall,  the  local  area  network, 
the  gateway  manager,  system  doctrine,  and  combat 
identification.  Additionally,  R3.1  lacked  adequate 
vulnerability  management  plans. 

A  final  assessment  of  R3.2  performance  will  not  be  available 

until  all  testing  is  completed  in  FY13  and  DOT&E  and  the  Air 

Force  have  analyzed  the  data.  However,  DOT&E  preliminary 

analyses  indicate: 

-  A  critical  deficiency  was  discovered  during  operational 
testing  at  the  Eastern  Air  Defense  Sector.  The  BCS-F 
system  did  not  reliably  pass  tracks  to  the  Joint  Air  Defense 
Operations  Center  at  Bolling  AFB,  Washington,  D.C.  This 
deficiency  causes  a  loss  of  situational  awareness  for  the 
operators  conducting  surveillance  of  the  National  Capital 
Region  and  results  in  an  inaccurate  air  picture.  The 
Eastern  Air  Defense  Sector  will  not  accept  R3.2  until  this 
deficiency  is  corrected.  The  Air  Force  planned  to  resolve 
this  deficiency  in  R3.2.1,  which  underwent  testing  from 
September  through  October  2012. 

-  R3.2. 1  did  not  resolve  the  critical  deficiency,  and  DOT&E 
assessed  it  as  not  effective  and  not  suitable.  The  Air  Force 
subsequently  developed  a  new  software  release  (R3.2.0.1), 
which  completed  testing  at  the  end  of  November; 
preliminary  data  shows  it  fixed  the  critical  deficiency. 
R3.2.0.1  also  successfully  demonstrated  a  temporary 
solution  to  operate  with  mandatory  International  Civil 
Aviation  Organization  flight  plan  changes.  The  permanent 
solution  will  be  incorporated  into  R3.2.2.  DOT&E  will 
update  the  assessment  of  R3.2  once  all  test  data  has  been 
received  and  analyzed. 

-  Operational  testing  of  R3.2  also  uncovered  the  potential  for 
BCS-F  to  provide  inaccurate  bearing  and  range  information 
on  an  air  track  of  interest.  The  effect  of  this  deficiency  is 
that  commanders  could  make  incorrect  tactical  decisions 
based  on  erroneous  data.  The  operational  sectors  assessed 
this  deficiency  as  critical,  but  Air  Combat  Command 
downgraded  the  severity  and  plans  to  implement  a  fix 

in  R3.2.2,  which  the  Air  Force  plans  to  start  testing  in 
January  2013. 

-  Test  personnel  discovered  six  other  critical  deficiencies 
during  R3.2  testing,  but  the  Air  Force  paused  testing, 
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developed  a  fix  for  each  critical  deficiency,  and 
demonstrated  each  fix  during  subsequent  test  events. 

-  The  BCS-F  is  required  to  demonstrate  a  Mean  Time 
Between  Critical  Failure  of  10,000  hours  with  an 
operational  availability  of  99.98  percent.  Six  critical 
failures  occurred  during  699  hours  of  operational  testing 
at  the  System  Support  Facility  and  the  Eastern  and 
Western  Air  Defense  Sectors.  Total  system  downtime 
at  the  System  Support  Facility  and  Eastern  and  Western 
Air  Defense  Sectors  was  approximately  44  hours  for  an 
operational  availability  of  90.5  percent  during  operational 
testing,  which  users  consider  inadequate  given  the  mission 
of  BCS-F.  However,  the  Air  Force  has  subsequently 
demonstrated  fixes  for  the  failures  causing  the  downtime. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
satisfactorily  addressed  all  but  three  of  the  previous 
recommendations.  The  Air  Force  still  needs  to: 

1.  Correct  and  formalize  all  BCS-F  Increment  3  system 
documentation  and  training  deficiencies. 


2.  Develop  a  plan  for  remote  workstation  management  to 
include  sustainment,  training,  documentation,  and  IA 
compliance. 

3.  Upgrade  the  System  Support  Facility  to  support  a  more 
robust  BCS-F  developmental  and  operational  testing 
capability  in  order  to  minimize  the  impact  of  overall  testing 
at  the  operational  sites. 

•  FY12  Recommendations.  The  Air  Force  should: 

1.  Improve  R3.2  reliability  and  operational  availability  until 
these  suitability  requirements  are  demonstrated  with 
statistical  confidence. 

2.  Continue  to  track  and  correct  1A  deficiencies. 

3.  Demonstrate  the  permanent  solution  for  R3.2  to 
successfully  incorporate  and  process  International  Civil 
Aviation  Organization  flight  plans. 

4.  Update  the  R3.2  Test  and  Evaluation  Master  Plan  to  reflect 
planning  and  FY13  funding  for  R3.2  follow-on  testing. 
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Executive  Summary 

•  In  February  2012,  the  Air  Force  completed  a  Force 
Development  Evaluation  of  Operational  Flight  Program 
(OFP)  3.5  software  combined  with  modifications  to  the 
engine  thrust  reversers.  The  C-5M  with  these  upgrades  is 
operationally  effective,  but  it  is  still  not  operationally  suitable. 

•  The  C-5  Program  Office  is  addressing  deficiencies  identified 
during  previous  operational  tests,  including  problems  with 
training  systems  and  devices,  Information  Assurance, 
additional  nuisance  faults,  the  mission  computers,  the 
automatic  throttles,  and  various  maintenance  Technical 
Orders  through  an  extensive  correction  action  plan.  The 
Air  Force  plans  to  begin  operational  testing  of  OFP  3.5.2  in 
August  2013. 

•  The  Air  Force  intends  to  award  a  contract  for  Core  Mission 
Computer  and  Weather  Radar  upgrades  in  FY13. 

System 

•  The  C-5  is  the  largest  four-engine  military  transport  aircraft 
in  the  United  States.  The  C-5  has  36  standard  463L  pallet 
positions  and  airline  seats  for  81  passengers.  It  can  carry 

a  maximum  payload  of  270,000  pounds.  The  typical  C-5 
crew  size  is  seven  (two  pilots,  two  flight  engineers,  and  three 
loadmasters). 

•  The  C-5M  designation  is  the  result  of  two  separate  but  related 
modernization  efforts: 

-  The  Avionics  Modernization  Program  incorporates 
a  mission  computer,  a  glass  cockpit  with  digital 
avionics  (including  autopilot  and  automatic  throttles), 
and  state-of-the-art  communications,  navigation,  and 
surveillance  components  for  air  traffic  management. 

-  The  Reliability  Enhancement  and  Re-Engining  Program 
provides  over  50  reliability  enhancements,  plus 
commercial  engines  with  new  nacelles,  thrust  reversers, 
and  pylons. 


Mission 

•  Units  equipped  with  the  C-5  perform  strategic  airlift, 
emergency  aeromedical  evacuation,  transport  of  brigade-size 
forces  and  equipment  in  conjunction  with  other  aircraft,  and 
delivery  of  outsize  or  oversize  cargo  (cargo  that  does  not  fit  on 
a  standard  pallet). 

•  Units  equipped  with  the  C-5  execute  missions  at  night,  in 
adverse  weather  conditions,  and  in  civil-controlled  air  traffic 
environments  around  the  world.  The  units  are  capable  of 
completing  extended-range  missions  because  the  C-5  can 
receive  in-flight  aerial  refueling. 

Major  Contractor 

Lockheed  Martin  Aeronautics  Company  -  Marietta,  Georgia 


Activity 

•  In  February  2012,  the  Air  Force  completed  a  Force 
Development  Evaluation  of  OFP  3.5  software  combined  with 
modifications  to  the  engine  thrust  reversers  in  accordance  with 
the  DOT&E-approved  test  plan.  OFP  3.5  is  the  second  major 
modification  to  C-5M  software. 

•  The  Air  Force  made  hardware  modifications  to  the  thrust 
reversers  to  address  known  deficiencies  with  deployment  and 
retraction  of  the  reversers. 

•  The  Air  Force  began  developmental  testing  of  the  next  C-5M 
software  version.  Block  3.5.2,  in  May  2012  and  plans  to 
conduct  operational  testing  in  August  2013. 


•  The  C-5  Program  Office  continues  to  address  deficiencies 
identified  during  the  C-5M  initial  operational  tests  in  2010, 
including  training  systems  and  devices,  Information 
Assurance,  additional  nuisance  faults,  the  mission  computers, 
the  automatic  throttles,  and  various  maintenance  Technical 
Orders  through  a  long-term  corrective  action  plan. 

•  The  Air  Force  intends  to  award  supplier  contracts  for  a  new 
Core  Mission  Computer  and  a  new  Weather  Radar  in  early 
FY13. 
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Assessment 

•  The  C-5M  is  operationally  effective  with  limitations.  OFP  3.5 
mitigates  Environmental  Control  System  Deficiency  Reports, 
reduces  nuisance  faults  related  to  the  Engine  Electronic 
Control,  and  marginally  improves  performance  of  the 
automatic  throttles. 

-  Operational  problems  remain  that  constrain  throttle  usage 
and  increase  pilot  workload,  including  the  automatic 
throttle’s  inability  to  “capture”  a  pre-set  airspeed  and 

the  extent  of  airspeed  deviations  during  critical  flight 
conditions,  such  as  final  approach.  OFP  3.5  modifications 
to  the  Environmental  Control  System  and  automatic 
throttles  mitigated  cabin  pressure  and  cabin  temperature 
fluctuations  and  reduced  the  very  large  throttle  movements 
associated  with  the  use  of  automatic  throttles.  However, 
failure  to  maintain  a  commanded  airspeed  during  critical 
phases  of  flight  remains  a  problem.  The  automatic  throttle 
response  improved  with  OFP  3.5  but  still  deviated 
outside  specified  criteria.  During  gusty  or  turbulent  wind 
conditions,  the  automatic  throttle  had  difficulty  maintaining 
the  set  speed. 

-  Prior  to  the  recent  modifications,  the  thrust  reversers 
did  not  deploy  reliably  in  flight  due  to  the  freezing  of 
condensed  water  within  the  Center  Drive  Units  of  the  thrust 
reversers.  This  condition  limited  the  aircraft’s  capability 
for  procedures  such  as  rapid  descent  from  high  altitude  and 
tactical  descent  in  a  combat  zone.  The  Air  Force  installed 

a  computer-controlled  heater  blanket  on  the  Center  Drive 
Units,  which  prevented  ice  formation  and  significantly 
increased  the  reliability  of  thrust  reverser  deployment  in 
flight. 

-  During  developmental  tests  of  the  heater  blanket,  the 
thrust  reversers  exhibited  retraction  problems.  If  the 
reversers  do  not  fully  retract,  the  likelihood  of  a  subsequent 
unintended  deployment  increases.  Therefore,  the  engineers 
increased  the  retraction  force  and  added  mechanical 
snubbers  (devices  used  to  absorb  excess  force)  to  the  thrust 
reversers.  Now,  the  reversers  deploy  and  retract  reliably  in 
flight,  but  require  additional  inspections. 

•  The  C-5M  is  still  not  operationally  suitable.  The  aircraft’s 
ability  to  conduct  the  strategic  airlift  mission  is  hindered 
by  deficiencies  in  the  Automatic  Flight  Control  System,  by 
problems  with  the  Embedded  Diagnostics  System  (EDS) 
and  built-in  test  (BIT)  functionality,  by  inadequate  support 
equipment,  and  by  a  lack  of  dedicated  training  systems. 


Deficiencies  in  several  aspects  of  C-5M  support  functions, 
identified  before  the  2010  OT&E  began,  had  a  significant 
effect  on  suitability,  specifically  the  maintainability  of  the 
aircraft.  Planned  fixes  for  the  deficiencies  described  below 
remain  in  development: 

-  BIT  -  a  very  high  false  alarm  rate  combined  with  a 
low  fault  isolation  rate  increased  the  time  needed  to 
troubleshoot  and  complete  maintenance  actions.  BIT 
detections  of  critical  faults  did  not  meet  the  requirement 
of  99  percent  during  operational  testing.  Incremental 
improvements  are  underway. 

-  Training  Systems  and  Devices  -  aircrew  and  maintainer 
training  devices  specific  to  the  C-5M  are  just  becoming 
available  two  years  after  the  Full-Rate  Production  decision. 
The  Air  Force  uses  one  simulator  and  on-aircraft  training 
to  mitigate  the  shortage  of  aircrew  simulators.  Maintainers 
still  have  insufficient  simulators  and  must  conduct 
training  on  the  aircraft,  which  is  restricted  by  the  aircraft 
availability.  The  Air  Force  intends  to  accredit  simulators 
for  both  aircrews  and  maintainers  in  FY13. 

-  Information  Assurance  -  the  C-5M  is  susceptible  to 
Information  Assurance  problems.  The  additional  risk  from 
information  operations  on  the  EDS  is  low.  Air  Mobility 
Command  is  addressing  the  Information  Assurance 
deficiencies  in  the  interface  of  the  EDS.  Improvements 
are  anticipated  in  the  Block  3.5.2  and  the  Core  Mission 
Computer  upgrades. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  is 
addressing  all  previous  recommendations. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Continue  to  pursue  comprehensive  remediation  programs 
for  known  capability  deferrals/deficiencies  in  the  C-5 
modernization  programs. 

2.  Aggressively  seek  timely  and  technically  accurate  aircrew 
and  maintenance  training  systems  and  documents  for  the 
C-5M. 

3.  Regularly  report  on  suitability  improvements  and  shortfalls 
in  both  new  and  legacy  C-5M  subsystems. 

4.  Correct  the  remaining  BIT,  EDS,  and  engine  support 
equipment  deficiencies. 
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Executive  Summary 

•  The  C-130J  is  in  production  with  periodic  Block  Upgrades  to 
correct  deficiencies  and  to  provide  capability  enhancements. 

•  Improvements  to  the  Station  Keeping  Equipment  (SKE), 
verified  in  a  second  FOT&E,  now  enable  the  C-130J 

to  perform  formation  airdrop  missions  in  instrument 
meteorological  conditions  (IMC),  correcting  an  effectiveness 
shortfall  from  the  C-130J  IOT&E  in  2005. 

•  The  Air  Force  conducted  FOT&E  of  the  Data  Transfer 
and  Diagnostics  System  (DTADS),  which  will  replace  the 
legacy  computerized  maintenance  system,  in  order  to  correct 
suitability  shortfalls  from  the  IOT&E. 

•  The  Air  Force  is  correcting  some  deficiencies  and  adding 
new  capabilities  in  the  Block  Upgrade  7.0.  Delivery  of  the 
upgrade  has  continued  to  experience  delays,  now  estimated  to 
occur  by  2QFY13.  The  Air  Force  has  not  funded  subsequent 
deployment  of  Block  Upgrade  7.0  to  the  fleet,  and  operational 
testing  of  the  upgrade  in  FY13  remains  undefined.  This 
deferment  affects  several  mission  design  series  of  the  C-130J 
operated  by  the  Air  Force,  Navy,  and  Coast  Guard. 

System 

•  The  C-130J  is  a  medium-sized  four-engine  turboprop  tactical 
transport  aircraft. 

•  Compared  to  previous  models,  the  cockpit  crew  requirement 
is  reduced  from  four  to  two  on  the  J  model;  loadmaster 
requirements  vary  (one  or  two),  depending  on  mission  need. 

•  Compared  to  legacy  models,  the  C-130J  has  approximately 
70  percent  new  development.  Enhancements  unique  to  the 
C-130J  include  a  glass  cockpit  and  digital  avionics,  advanced 
integrated  diagnostics,  a  new  propulsion  system,  improved 
defensive  systems,  and  an  enhanced  cargo  handling  system. 

•  The  C-130J  has  two  different  lengths  denoted  as  a  long  and  a 
short  body.  The  long  body  carries  eight  standard  pallets;  the 
short  carries  six. 


Mission 

•  Combatant  Commanders  use  the  C-130J  within  a  theater  of 
operations  for  combat  delivery  missions  which  include: 

-  Airdrop  of  paratroopers  and  cargo  (palletized, 
containerized,  bulk,  and  heavy  equipment) 

-  Airland  delivery  of  passengers,  troops,  and  cargo 

-  Emergency  aeromedical  evacuations 

•  Combat  Delivery  units  operate  in  all  weather  conditions,  use 
night-vision  lighting  systems,  and  may  be  required  to  operate 
globally  in  civil-controlled  airspace. 

Major  Contractor 

Lockheed  Martin  Aeronautics  Corporation  -  Fort  Worth,  Texas 


Activity 

•  The  Air  Force  is  correcting  deficiencies  found  in  both 
developmental  and  operational  testing  and  adding  new 
capabilities  in  the  Block  Upgrade  7.0.  Block  Upgrade  7.0 
continues  to  experience  delays,  with  government 
developmental  test  and  evaluation  now  expected  to  be 
complete  by  2QFY13.  The  Air  Force  is  deferring  some 
deficiencies  originally  planned  for  correction  in  Block 
Upgrade  7.0  to  Block  Upgrade  8.1.  The  Air  Force  has  not 
funded  the  deployment  of  Block  Upgrade  7.0,  which  will 
likely  become  the  technical  baseline  for  the  development  of 
Block  Upgrade  8.1.  The  Air  Force  has  not  fully  defined  plans 


for  operational  testing  of  Block  Upgrade  7.0  and  may  limit 
operational  testing  to  an  operational  assessment  of  limited 
capability  enhancements  in  3QFY13. 

•  The  Air  Force  conducted  FOT&E  of  the  DTADS  in  October 
and  November  2011  in  accordance  with  the  DOT&E-approved 
test  plan.  DTADS  will  replace  the  current  computerized 
maintenance  system  (the  integrated  diagnostics  system 
interface  and  Portable  Maintenance  Aid)  that  had  suitability 
shortfalls  during  the  C-130J  IOT&E.  DOT&E  released  a 
report  on  the  DTADS  FOT&E  in  October  2012. 
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•  The  Air  Force  conducted  a  second  FOT&E  of  the  SKE 
software  enhancement  from  January  to  March  2012  in 
accordance  with  the  DOT&E-approved  test  plan.  DOT&E 
released  reports  on  the  first  FOT&E  in  February  2012  and  the 
second  FOT&E  in  October  2012. 

Assessment 

•  The  Air  Force  decision  not  to  deploy  Block  Upgrade  7.0  will 
affect  all  mission  design  series  C-130J  aircraft,  including  Air 
Force  EC-,  HC-,  MC-,  and  WC-130J;  Navy  KC-130J;  and 
Coast  Guard  HC-130J.  Some  future  aircraft  acquisitions  that 
planned  to  incorporate  Block  Upgrade  7.0  modifications  in  the 
production  line  will  have  to  be  re-planned  as  kit  installations  at 
an  increased  cost  and  decreased  aircraft  availability. 

•  DTADS  represents  a  significant  improvement  over  the  legacy 
maintenance  support  system  in  terms  of  usability,  portability, 
diagnostic  capability,  and  organic  maintainability.  Information 
Assurance  shortfalls  remain,  particularly  concerning  the 
Windows  XP®  operating  system  used  by  DTADS  and  the 
procedures  for  handling  potentially  classified  digital  flight 
data  recorder  information.  The  Air  Force  cannot  connect 
DTADS  to  government  computer  networks  without  a  required 
Windows  7®  update. 


•  The  SKE  has  demonstrated  significant  improvement  in 
successfully  enabling  formation  flight  in  IMC,  and  some 
improvement  in  reliability.  However,  shortfalls  remain  in 
the  troubleshooting  and  repair  of  SKE  faults.  The  Air  Force 
restricts  formation  flight  in  IMC  to  formations  of  only  C-130J 
aircraft,  limiting  interoperability  with  legacy  C-130  models. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has  made 
sufficient  progress  in  addressing  the  FY 1 1  recommendation 

to  correct  deficiencies  in  formation  flying  that  will  enable  the 
SKE  capability  release.  However,  maintainability  shortfalls 
remain  in  the  troubleshooting  and  repair  of  SKE  faults. 

•  FY12  Recommendations.  The  Air  Force  should: 

1.  Correct  remaining  deficiencies  in  the  technical  data 
concerning  the  operation,  fault  isolation,  and  repair  of  the 
SKE  system. 

2.  Finalize  plans  for  FY13  operational  testing  of  Block 
Upgrade  7.0  and  communicate  them  to  DOT&E. 

3.  Ensure  adequate  training  and  protocols  are  established  for 
managing  classified  media  associated  with  DTADS  and 
digital  flight  data  recorder  downloads. 
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Defense  Enterprise  Accounting  and 
Management  System  (DEAMS) 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  an  Operational  Assessment  (OA) 
of  the  Defense  Enterprise  Accounting  and  Management 
System  (DEAMS)  Increment  1  Release  1  from  May  through 
June  2012.  The  test  locations  included  Headquarters  U.S. 
Transportation  Command  (HQ  USTRANSCOM)  and 
Headquarters  Air  Mobility  Command  (HQ  AMC),  both 
located  at  Scott  AFB,  Illinois;  and  the  Defense  Finance  and 
Accounting  Service  in  Limestone,  Maine.  An  Air  Force  team 
from  the  688th  Information  Operations  Wing  conducted  a 
limited  Information  Assurance  (IA)  vulnerability  assessment 
at  Maxwell-Gunter  Annex,  Alabama,  on  June  26,  2012. 

•  DEAMS  is  neither  operationally  effective  nor  operationally 
suitable.  Many  key  features  do  not  work,  while  other  features 
often  require  intervention  by  subject  matter  experts  to  make 
them  work. 

•  DEAMS  successfully  performs  some  decision  support 
functions  such  as  budget  management  within  authority 
and  funds  distribution  and  loading.  However,  it  does  not 
adequately  perform  budget  analysis  and  planning,  decision 
analysis,  balancing  the  General  Ledger,  not  exceeding  budget 
control  targets,  and  generating  accurate  reports.  Many  users 
rely  on  legacy  systems  to  create  reports. 

•  DEAMS  is  neither  making  sufficient  progress  towards 
achieving  audit  readiness  of  the  Statement  of  Budgetary 
Resources  by  the  end  of  2014,  nor  towards  achieving  full 
financial  auditability  by  the  statutory  deadline  of  September 
30,  2017. 

•  DEAMS  could  already  be  the  target  of  fraud  or  theft  due 
to  inconsistent  telecommuting  policies,  1A  vulnerabilities, 
inadequate  control  of  budget  targets,  and  large  account 
imbalances. 

•  DEAMS  is  unable  to  match  Accounts  Payable  or  Accounts 
Receivable,  reconcile  subsidiary  accounts  to  the  General 
Ledger  (none  reconciled),  perform  end-of-year  accounting 
closeouts,  or  balance  appropriation  accounts  accurately 
with  Treasury  appropriation  accounts.  However,  DEAMS 
successfully  ages  and  liquidates  Accounts  Receivable, 
capitalizes  and  depreciates  assets  when  full  information  is 
available,  and  processes  only  valid  data. 

•  DEAMS  does  not  provide  timely  information  to  close  out 
financial  accounts  at  the  designated  end-of-period  times; 
however,  it  records  transactions  quickly  and  pays  vendors 
promptly. 

•  The  presence  of  over  200  high-severity  operational 
deficiencies  indicates  that  the  software  is  unstable.  Many 
of  the  deficiencies  are  related  to  interfaces,  indicating 
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that  DEAMS  is  not  satisfying  the  end-to-end  information 
exchange  requirements  of  the  Net-Ready  Key  Performance 
Parameter  (KPP).  However,  DEAMS  hardware  is  reliable  and 
response  times  are  acceptable. 

•  Three  of  seven  previously-identified  major  IA  vulnerabilities 
still  require  correction. 

•  DEAMS  users  are  highly  critical  of  user  training  and  user 
documentation. 

System 

•  DEAMS  is  a  Major  Automated  Information  System  that 
uses  commercial  off-the-shelf  Enterprise  Resource  Planning 
software  to  provide  accounting  and  management  services. 

•  The  Program  Management  Office  (PMO)  is  following 
an  evolutionary  acquisition  strategy  that  adds  additional 
capabilities  incrementally.  DEAMS  Increment  1  consists 
of  six  releases.  Release  1,  which  provides  approximately 
75  percent  of  the  system’s  capability,  is  designed  to  meet 
the  accounting  and  financial  management  needs  of  HQ 
USTRANSCOM  and  HQ  AMC.  Releases  2  through  6 
intend  to  provide  enhanced  capabilities  to  support  other 
USTRANSCOM  locations. 

•  DEAMS  operates  on  the  Global  Combat  Support 
System  -  Air  Force  Integration  Framework.  It  interfaces 
with  approximately  40  other  systems  that  provide  travel, 
payroll,  disbursing,  transportation,  logistics,  acquisition,  and 
accounting  support. 

Mission 

•  USTRANSCOM  and  Air  Force  financial  managers  will  use 
DEAMS  to  compile  and  share  accurate,  up-to-the-minute 
financial  management  data  and  information  across 
USTRANSCOM  and  the  Air  Force. 

•  USTRANSCOM,  Air  Force,  and  DoD  leadership  will  use 
DEAMS  to  access  vital,  standardized,  real-time  financial  data 
and  information  to  make  strategic  business  decisions. 
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•  USTRANSCOM  and  the  Air  Force  will  use  DEAMS  to  satisfy  Major  Contractor 

congressional  and  DoD  requirements  for  auditing  funds,  Accenture  Federal  Services  -  Fairborn,  Ohio 

standardizing  financial  ledgers,  timely  reporting,  and  reduction 
of  costly  rework. 


Activity 

•  AFOTEC  began,  but  did  not  complete,  an  Early  Operational 
Assessment  (EOA)  of  DEAMS  from  August  through 
December  2010  in  accordance  with  a  DOT&E-approved  test 
plan.  AFOTEC  cut  the  EOA  short  when  it  became  apparent 
that  major  system  deficiencies  were  present  and  that  the 
planned  Milestone  B  was  significantly  delayed. 

•  After  the  program  manager  declared  that  DEAMS  had  been 
stabilized,  AFOTEC  conducted  a  second  OA  of  DEAMS 
Increment  1  Release  1  from  May  through  June  2012  in 
accordance  with  a  DOT&E-approved  test  plan.  As  with 
the  EOA,  the  test  locations  included  HQ  USTRANSCOM 
and  HQ  AMC,  both  located  at  Scott  AFB,  Illinois;  and  the 
Defense  Finance  and  Accounting  Service  in  Limestone,  Maine. 
DEAMS  Increment  1  Release  1  contained  approximately 

75  percent  of  its  final  capabilities  and  had  been  in  operational 
use  for  two  years. 

•  An  Air  Force  team  from  the  688th  Information  Operations 
Wing  conducted  a  limited  IA  vulnerability  assessment  at 
Maxwell-Gunter  Annex,  Alabama,  in  June  2012. 

•  In  July  2012,  the  DoD  Inspector  General  reported 
(DoDIG-2012-111)  that  DEAMS ’s  original  life-cycle  cost 
estimate  had  more  than  quintupled  to  over  $2  Billion  and  that 
the  program  is  7.5  years  behind  schedule.  In  September  2012, 
another  DoD  Inspector  General  report  (DoDIG-2012-140) 
found  that  DEAMS  lacked  critical  functional  capabilities 
needed  to  generate  accurate  and  reliable  financial  management 
information  and  might  not  meet  its  2014  and  2017  audit 
readiness  requirements. 

•  DOT&E  issued  an  OA  report  on  DEAMS  in  September  2012. 
A  Milestone  Decision  Authority  decision  followed 
immediately  after,  allowing  the  Air  Force  to  deploy  DEAMS 
to  an  additional  150  AMC  users  at  McConnell  AFB,  Kansas. 

Assessment 

•  AFOTEC  conducted  the  FY12  OA  of  DEAMS  Increment  1 
Release  1  with  the  rigor  of  an  IOT&E. 

•  DEAMS  is  neither  operationally  effective  nor  operationally 
suitable.  Many  key  features  do  not  work,  while  other  features 
often  require  intervention  by  subject  matter  experts  to  make 
them  work. 

•  Operational  testers  documented  deficiencies  in  the  areas  of 
information  quality,  timeliness,  and  usability,  along  with  many 
undocumented  workarounds.  DEAMS  hardware  is  reliable 
and  response  times  are  acceptable,  but  the  presence  of  more 
than  200  unresolved,  high-severity  deficiencies  indicates  that 
the  software  is  still  unstable,  despite  program  attempts  to 


resolve  245  prior  high  priority  defects.  According  to  the  PMO, 
the  fast  tempo  of  delivery  of  software  patches  has  not  left  any 
time  for  regression  testing. 

•  Many  of  the  software  deficiencies  are  related  to  interfaces, 
indicating  that  DEAMS  is  not  satisfying  the  end-to-end 
information  exchange  requirements  of  the  Net  Ready  KPR 

•  DEAMS  is  neither  making  sufficient  progress  towards 
achieving  audit  readiness  of  the  Statement  of  Budgetary 
Resources  by  the  end  of  2014,  nor  towards  achieving 
full  financial  auditability  by  the  statutory  deadline  of 
September  30,  2017. 

•  DEAMS  is  unlikely  to  support  financial  operations  because 
it  does  not  meet  the  KPP  for  appropriations  balance  with  the 
U.S.  Treasury.  Of  723  appropriation  accounts  examined, 

109  had  amounts  out  of  balance  for  a  total  of  $449  Million. 
DEAMS  also  does  not  meet  the  95  percent  KPP  for  General 
Ledger  to  subsidiary  ledger  reconciliation  (none  of  the  49 
samples  reviewed  were  reconciled). 

•  DEAMS  is  unable  to  accurately  match  any  Accounts  Payable 
or  Accounts  Receivable  documents  and  does  not  always 
prevent  the  posting  of  transactions  that  exceed  Congressional 
appropriation  funds  control  targets.  AFOTEC  identified 
two  transactions  that  exceeded  strict  funds  control  by  a  total 
of  $3  Million.  However,  DEAMS  successfully  ages  and 
liquidates  Accounts  Receivable,  capitalizes  and  depreciates 
assets  when  full  information  is  available,  and  processes  only 
valid  data. 

•  DEAMS  does  not  meet  the  KPP  for  timely  reporting  of 
period-end  data  to  the  official  reporting  system.  DEAMS  has 
not  properly  performed  the  end-of-year  accounting  closeouts 
for  FY10  or  FY11;  however,  it  records  transactions  quickly 
and  pays  vendors  promptly. 

•  DEAMS  does  not  sustain  financial  decision  support  because 
it  does  not  meet  the  KPP  for  fully  accurate  status  of  funds. 
While  DEAMS  successfully  performs  some  decision  support 
functions  such  as  budget  management  within  authority  and 
funds  distribution  and  loading,  several  important  reports  were 
inaccurate  or  incomplete.  Many  operators  continue  to  use 
legacy  systems,  rather  than  DEAMS,  to  perform  their  daily 
tasks. 

•  DEAMS  does  not  meet  the  KPP  for  budget  control  targets  and 
is  susceptible  to  fraud  or  theft.  The  additional  in-test  discovery 
of  inconsistent  telecommuting  policies,  IA  vulnerabilities,  and 
large  account  imbalances  raise  a  concern  that  fraud  or  theft 
may  be  taking  place. 
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•  Three  of  seven  previously-noted  major  IA  vulnerabilities  still 
require  correction.  Furthermore,  some  DEAMS  users  are 
telecommuting  without  using  a  Virtual  Private  Network. 

•  DEAMS  users  are  highly  critical  of  user  training  and  user 
documentation. 

Recommendations 

•  Status  of  FY1 1  Recommendations.  The  PMO  and  AFOTEC 
addressed  all  FY 1 1  recommendations. 

•  FY12  Recommendations. 

1.  The  Air  Force  should  conduct  immediate  “Financial 
Red  Team”  penetration  testing  to  assess  DEAMS ’s  fraud 
and  theft  vulnerabilities.  The  test  should  address  both 
prevention  and  detection  of  potential  thefts  or  frauds. 

2.  The  Air  Force  should  conduct  immediate  tests  of  the  1A 
risks  associated  with  telecommuting.  The  tests  should 
determine  whether  perpetrators  have  attacked  the  system 
and  extracted  data.  The  PMO  and  the  Defense  Information 
Systems  Agency  should  rectify  all  remaining  1A  deficiencies 
in  order  to  reduce  vulnerability. 


3.  In  order  to  prevent  the  accrual  of  new  deficiencies,  the 
Air  Force  should  recreate  a  realistic  developmental  test 
environment  that  incorporates  automated  testing  and  allows 
the  conduct  of  thorough,  systematic,  repeatable  regression 
testing. 

4.  The  PMO  and  Functional  Management  Office  should 
document  any  required  workarounds  authorized  by  the 
users.  Similarly,  the  PMO  should  develop  training  to  better 
meet  individual  user  group  requirements  and  improve  user 
documentation  to  include  workarounds  and  the  legacy 
systems  that  are  still  needed  for  mission  accomplishment. 

5.  AFOTEC  and  the  Joint  Interoperability  Test  Command 
should  collaborate  to  assess  interoperability  that  covers  full 
Net-Ready  KPP  requirements,  including  the  end-to-end 
functional  exchange  of  required  information. 

6.  The  Air  Force  should  not  permit  further  deployments  of 
DEAMS  until  substantial  progress  is  made  addressing  the 
system’s  deficiencies. 
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E-3  Airborne  Warning  and  Control  System  (AWACS) 


Executive  Summary 

•  The  E-3  Airborne  Warning  and  Control  System  (AWACS) 
provides  airborne  early  warning,  air  surveillance,  air  battle 
management,  and  command  and  control  without  the  beyond 
line-of-sight  limitations  inherent  in  ground-based  air  battle 
management  systems. 

•  The  Block  40/45  upgrade  replaces  the  mission  computing 
system  on  the  E-3  with  open-architecture,  commercial 
off-the-shelf  hardware  including  servers,  and  15  mission  crew 
interactive  displays. 

•  The  primary  combat  capability  provided  by  the  increased 
processing  power  of  the  Block  40/45  mission  computing 
upgrade  is  to  automatically  fuse  all  onboard  and  off-board 
sensor  inputs  to  provide  a  single  track  with  a  fused 
identification  for  each  air,  sea,  and  land  entity  using  a 
multi-sensor  integration  algorithm. 

•  Block  40/45  is  operationally  effective.  The  modification 
provides  many  improvements  for  the  operators,  including 
automated  tracking  and  identification.  However,  it  did  not 
provide  the  operators  adequate  control  of  the  automated 
tracking  capability.  Additionally,  it  does  not  provide 
required  enhancements  to  battle  management  capabilities, 
specifically  the  ability  to  automatically  import  data  from  the 
Air  Operations  Center  to  update  the  onboard  database.  It  also 
does  not  provide  Block  30/35  equivalent  Link  16  datalink 
capabilities. 

•  Block  40/45  is  not  operationally  suitable.  During  the 
IOT&E,  the  E-3  arrived  on-station,  on-time  with  both  the 
radar  and  the  Identification  Friend  or  Foe  (IFF)  interrogation 
system  functioning  only  one-third  of  the  time.  Operator  and 
maintainer  training  were  deficient  and  repair  times  exceeded 
requirements.  Although  Block  40/45  does  not  currently  meet 
several  key  suitability  requirements,  Block  40/45  hardware 

is  more  reliable  than  the  aging  Block  30/35  equipment 
it  replaces.  Even  when  software  failures  are  included, 

Block  40/45  is  still  more  reliable  than  Block  30/35. 

System 

•  AWACS  is  built  on  a  Boeing  707  airframe.  A  surveillance 
radar  and  IFF  system  are  located  in  the  rotodome  above 
the  airframe.  An  Electronic  Support  Measures  system  has 
antennas  on  the  cheeks  of  the  airframe,  under  the  nose,  and  in 
the  tail.  The  E-3  has  13  Ultra  High  Frequency  radios,  4  Very 
High  Frequency  radios,  and  3  High  Frequency  radios. 

•  The  Block  40/45  upgrade  replaces  the  mission  computing 
system  on  the  E-3  with  open-architecture,  commercial 
off-the-shelf  hardware  including  servers,  and  15  mission 
crew  interactive  displays.  The  mission  computing  software 
program  is  replaced  with  a  set  of  local  area  networked, 
open-architecture  programs.  The  human-computer  interface 


is  built  on  the  Windows®  operating  system  and  licenses  the 
Raytheon  Solipsys©  Tactical  Display  Framework. 

•  The  Block  40/45  mission  computing  upgrade  provides  the 
capability  to  automatically  fuse  all  onboard  and  off-board 
sensor  inputs  to  provide  a  single  track  for  each  air,  sea,  and 
land  entity  using  a  multi-sensor  integration  algorithm.  The 
upgrade  is  also  intended  to  provide: 

-  An  update  to  the  E-3  AWACS  Link  16  and  satellite 
communications  capabilities 

-  Software  to  automatically  refresh  the  onboard  database 

-  An  updated  mission  system  health  monitoring  tool 

-  Improved  interfaces  and  controls  of  the  onboard  passive 
Electronic  Support  Measure  system 

-  Improved  mission  planning  and  post  mission  processing 
capabilities 

•  The  first  six  Block  40/45  E-3s  are  planned  to  have  three 
different  mission  computing  configurations.  The  Air  Force 
plans  to  use  the  configuration  of  the  seventh  Block  40/45  E-3 
to  upgrade  the  next  1 1  jets. 

•  Block  40/45  requires  several  new  ground  support  systems 
including  the  mission  planning  system,  which  the  contractor 
delivered  with  the  first  upgraded  aircraft.  The  contractor  will 
deliver  a  deployable  mission  planning  system  before  Initial 
Operational  Capability  in  December  2013,  in  addition  to 
trainers  for  maintenance  personnel  and  mission  crew. 

Mission 

The  Air  Component  Commanders  use  AWACS-equipped  units  to: 

•  Provide  early  warning,  air  surveillance,  air  battle  management, 
and  beyond  line-of-sight  capabilities 

•  Provide  command  and  control  of  offensive  and  defensive 
counterair  and  countersea  operations,  and  strike  missions 
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including  dynamic  targeting,  close-air  support,  suppression  of  Major  Contractor 

enemy  air  defenses,  and  strategic  attacks  The  Boeing  Company  -  Seattle,  Washington 

•  Manage  air  refueling  operations,  combat  search  and  rescue 
missions,  and  special  operations  missions 


Activity 

•  During  F Y 1 1 ,  Boeing  conducted  developmental  test  and 
evaluation  (DT&E)  using  the  first  production  Block  40/45 
E-3.  Air  Force  Joint  Task  Force  personnel  were  present 
at  all  qualification  test  events  and  had  signature  authority 
for  pass/ fail  determination.  This  contractor-conducted 
DT&E  consisted  of  seven  flights  from  Boeing  Field,  Seattle, 
Washington.  The  focus  of  the  final  phase  of  DT&E  was  to 
verify  changes  made  to  the  production  system  after  the  Air 
Force  Operational  Test  and  Evaluation  Center  (AFOTEC) 
published  the  2007  Operational  Assessment  in  support  of  the 
Low-Rate  Initial  Production  decision. 

•  AFOTEC  conducted  a  24-flight  IOT&E  operating  from  the  E-3 
main  operating  base,  Tinker  AFB,  Oklahoma  City,  Oklahoma, 
between  March  and  June  2012.  The  IOT&E  was  conducted 

in  accordance  with  the  DOT&E-approved  test  plan.  While 
there  were  no  overnight  deployments,  the  two  operational 
Block  40/45  E-3s  participated  in  several  large  force  exercises. 
The  Block  40/45  IOT&E  included  flights  working  with  assets 
from  all  four  Services  in  training  areas  on  both  coasts  as  well 
as  over  land. 

Assessment 

•  Block  40/45  testing  was  adequate  to  support  an  evaluation  of 
operational  effectiveness  and  suitability. 

•  Block  40/45  was  not  ready  to  enter  IOT&E. 

Contractor-conducted  DT&E  focused  on  specification 
compliance  verification  in  lieu  of  a  government-conducted 
DT&E,  which  could  have  assessed  risks  to  a  successful 
IOT&E  outcome. 

•  Training  was  not  representative  of  the  syllabus  intended  for 
maintainers  and  aircrews.  Operator  and  maintainer  training 
simulators  were  not  ready  for  IOT&E.  The  deployable 
mission  planning  system  was  also  not  available  for  the 
IOT&E.  Additionally,  documentation  for  both  operators  and 
maintainers  was  incomplete. 

•  Several  Block  40/45  capabilities,  including  the  mission 
planning  system  and  start-up  checklist,  were  never  tested  in 
DT&E.  The  Program  Office  never  documented  workarounds 
for  use  by  aircrews  during  contractor  DT&E,  nor  did  they 
modify  the  system  design  to  reflect  changes  in  interoperability 
standards  during  Block  40/45  development. 

•  Block  40/45  is  operationally  effective.  It  provides  some 
improvements  for  the  operators,  but  not  all  the  required 
enhancements.  Block  40/45  provided  automated  tracking 
and  combat  identification,  but  did  not  provide  the  operators 
adequate  control  of  the  automated  tracking  capability.  The 
crews  were  able  to  accomplish  their  battle  management 


command  and  control  missions  throughout  the  IOT&E; 
however.  Block  40/45  did  not  adequately  provide  the  required 
capability  to  receive  free-text  data  from  the  Air  Operations 
Center  and  automatically  import  the  data  into  onboard 
databases. 

•  Block  40/45  does  not  provide  equivalent  Link  16  capabilities 
to  Block  30/35,  which  it  replaces.  AFOTEC  discovered 
several  interoperability  deficiencies  during  the  IOT&E.  Many 
of  the  tactical  datalink  deficiencies  were  caused  by  the  Air 
Force  not  modifying  the  system  design  to  reflect  changes  in 
interoperability  standards  during  Block  40/45  development. 
The  satellite  communications  terminal  did  not  provide  an 
operationally  useful  capability  to  receive  digital  information. 

•  Block  40/45  is  not  operationally  suitable.  During  the  IOT&E, 
the  Block  40/45  E-3  arrived  on-station,  on-time,  with  both  the 
radar  and  IFF  interrogation  system  functioning  only  one-third 
of  the  time.  Block  40/45  system  deficiencies  caused  half  the 
missed  on-station  times,  while  legacy  system  deficiencies 
caused  the  other  half. 

•  Block  40/45  demonstrated  poor  reliability.  The  Mean  Time 
Between  Critical  Failure  was  9  hours,  which  is  significantly 
less  than  the  threshold  of  2,500  hours.  Two-thirds  of  all 
critical  failures  occurred  while  starting  the  mission  computing 
system.  However,  the  Block  40/45  hardware  is  already  more 
reliable  than  the  Block  30/35  hardware  it  replaces  (72-hour 
vice  4.6-hour  Mean  Time  Between  Unscheduled  Maintenance 
(MTBUM)). 

•  When  considering  the  addition  of  software  failures  and  the 
reduction  of  Block  40/45  MTBUM,  Block  40/45  is  still  more 
reliable  than  Block  30/35. 

•  Two  ground  repair  actions  were  incomplete  at  the  end  of  the 
IOT&E.  One  open  repair  action  was  for  a  legacy  Block  30/35 
part.  The  second  open  repair  action  was  for  a  Block  40/45  part 
that  was  either  not  ordered  or  not  provided  for  14  days. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Incorporate  the  most  current  datalink  message  standards 
into  Block  40/45.  This  will  allow  Block  40/45  to  have 

a  datalink  capability  equivalent  to  the  fielded  legacy 
Block  30/35  AWACS  fleet. 

2.  Complete  and  update  aircrew  and  maintenance  checklists 
and  technical  orders  to  address  the  new  failure  modes 
discovered  during  IOT&E. 
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3.  Modify  the  mission  computing  software  and  refine 
technician  training  to  reduce  the  incidence  of  induced  critical 
failures  during  Block  40/45  mission  computing  startup. 

4.  Develop  software  modifications  to  improve  aircrew  ability  to 
control  the  automated  tracking  capability. 

5.  Review  and  update  the  planned  training  syllabus  for  both 
aircrew  and  maintenance  personnel  with  information  learned 
during  the  IOT&E. 


6.  Conduct  FOT&E  of  Block  40/45  using  the  first  Block  40/45 
configuration  that  will  be  installed  on  more  than  two 
aircraft.  The  FOT&E  should  include  an  operationally 
representative  deployment  in  a  stressful  tracking  and 
combat  identification  environment. 
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F-15E  Radar  Modernization  Program  (RMP) 


Executive  Summary 

•  F-15E  Radar  Modernization  Program  (RMP)  developmental 
flight  testing  began  in  January  2011  and  IOT&E  was  expected 
to  begin  in  late  FY12.  Although  the  RMP  continued  to  make 
progress  towards  operational  effectiveness,  suitability,  and 
mission  capability  during  FY12,  unanticipated  software 
performance  challenges  in  air-to-ground  mapping,  precision 
velocity  update,  and  air-to-air  target  detection  and  tracking  led 
to  developmental  test  delays.  As  a  result,  the  program  will  not 
begin  IOT&E  until  mid-FY13. 

•  The  Air  Force  resolved  electromagnetic  interference  between 
the  APG-82(V)1  Active  Electronically  Scanned  Array  (AESA) 
transmitter/receiver  and  the  radios  through  the  incorporation 
of  software  changes. 

•  The  Air  Force  made  progress  in  overcoming  aircraft 
Environmental  Control  System  (ECS)  component  failures  and 
in-flight  cautions  noted  in  F Y 1 1  flight  testing.  RMP  hardware 
performed  nominally  throughout  FY12  developmental  testing. 

System 

•  The  F-15E  is  a  twin  engine,  tandem  seat,  fixed  wing,  all 
weather,  multi-role  fighter  aircraft.  The  F-15E  has  a  fully 
missionized  cockpit  and  a  multimode  air  intercept  and 
air-to-ground  radar,  giving  the  aircrew  the  capability  to 
employ  air-to-air  and  air-to-ground  munitions,  a  20-millimeter 
cannon,  and  countermeasures  for  evading  enemy  fire. 

•  The  RMP  replaces  the  F-15E  legacy  APG-70  mechanically 
scanned  radar  with  an  AESA  system  designated  the 
APG-82(V)1.  The  RMP  is  designed  to  retain  functionality 
of  the  legacy  radar  system  while  providing  expanded  mission 
employment  capabilities  to  include: 

-  Near-simultaneous  interleaving  of  selected  air-to-air  and 
air-to-ground  functions 

-  Enhanced  air-to-air  and  air-to-ground  combat  identification 
capabilities 

-  Longer  range  air-to-air  target  detection  and  enhanced  track 
capabilities 

-  Longer  range  and  higher  resolution  air-to-ground  radar 
mapping 

-  Improved  ground  moving  target  track  capability 

•  The  RMP  upgrade  is  also  intended  to  address  legacy  F-15E 
radar  system  suitability  shortfalls  including:  poor  reliability, 


parts  obsolescence,  and  high  sustainment  costs.  The  Air  Force 
intends  to  retrofit  the  RMP  across  the  existing  F-15E  fleet. 

•  The  RMP  APG-82(V)1  design  leverages  capabilities  from 
currently  fielded  AESA  radar  systems.  The  APG-82(V)1 
antenna  and  power  supply  are  currently  in  use  on  the  F-15C 
APG-63(V)3  program,  and  the  radar  receiver/exciter  and 
Common  Integrated  Sensor  Processor  are  based  on  the 
F/A-18E/F  APG-79  AESA  system. 

•  Other  hardware  and  software  modifications  comprising  the 
RMP  effort  include  a  more  powerful  ECS,  updates  to  the 
aircraft  Operational  Flight  Program  and  Electronic  Warfare 
software,  a  new  radio  frequency  tunable  filter,  and  aircraft 
modifications  to  include  a  new  wideband  radome  and  wiring 
changes. 

Mission 

A  unit  equipped  with  the  F-15E  conducts  all  weather,  day  and 

night  missions  to  include: 

•  Offensive  and  Defensive  Counterair 

•  Conventional  Air  Interdiction  and  Nuclear  Strike 

•  Close  Air  Support  and  Strike  Coordination  and 
Reconnaissance 

•  Suppression  of  Enemy  Air  Defenses 

•  Combat  Search  and  Rescue 

Major  Contractors 

•  The  Boeing  Company  -  Saint  Louis,  Missouri 

•  Raytheon  -  El  Segundo,  California 


Activity 

•  The  Air  Force  conducted  F-15E  RMP  testing  in  accordance 
with  the  DOT&E-approved  Test  and  Evaluation  Master  Plan. 

•  Developmental  flight  testing  begun  in  January  2011  extended 
throughout  FY12.  The  planned  FY12  IOT&E  start  did  not 


occur  due  to  challenges  in  maturing  system  software  to  meet 
the  user’s  functional  requirements. 

•  The  Air  Force  resolved  electromagnetic  interference  between 
the  APG-82(V)1  AESA  transmitter/receiver  and  the  radios 
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through  the  incorporation  of  software  changes.  The  Air 
Force  verified  the  effectiveness  of  the  corrective  actions  in 
subsequent  FY12  developmental  flight  testing. 

•  The  program  experienced  developmental  challenges  in 
maturing  the  integrated  air-to-air  and  air-to-ground  capabilities 
throughout  FY12.  Unanticipated  software  performance 
challenges  in  air-to-ground  mapping,  precision  velocity 
update,  and  air-to-air  target  detection  and  tracking  led  to 
development  delays.  The  Air  Force  has  rescheduled  IOT&E 
for  March  through  J Lily  2013. 

Assessment 

•  The  RMP  continued  to  demonstrate  incremental  progress 
towards  achieving  the  system’s  operational  performance 
and  suitability  goals  during  FY12  developmental  testing. 
Flowever,  the  program  experienced  software  maturation 
challenges  and  was  unable  to  complete  developmental  testing 
in  FY12.  Unanticipated  software  performance  shortfalls  led  to 
multiple  radar  OFP  releases  and  associated  regression  testing 
to  mature  radar  mode  functionality.  At  the  end  of  FY12,  RMP 
performance  had  not  yet  met  the  user’s  requirements. 

•  Progress  was  made  in  overcoming  aircraft  ECS  component 
failures  and  in-flight  cautions  noted  in  FY1 1  flight  testing. 


Additionally,  RMP  hardware  performed  nominally  throughout 
FY12. 

•  As  noted  in  the  DOT&E  FY1 1  Annual  Report,  achieving  the 
Air  Force  RMP  software  stability  requirement  by  IOT&E  may 
not  be  feasible.  Multiple  radar  OFP  version  releases  to  correct 
system  functionality  shortfalls  throughout  FY12  precluded 
focus  on  stability  requirements.  As  highlighted  in  the  F  Y 1 1 
Annual  Report,  DOT&E  continues  to  assess  that  the  Air  Force 
is  not  likely  to  meet  the  30-hour  Mean  Time  Between  Software 
Anomaly  requirement. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
continues  to  address  all  previous  recommendations. 

•  FY12  Recommendation. 

1 .  The  Air  Force  should  consider  either  amending  the 
RMP  30-hour  Mean  Time  Between  Software  Anomaly 
requirement  or  structuring  the  program  (in  particular, 
adding  time  and  resources  for  additional  development)  such 
that  it  is  able  to  achieve  the  desired  performance  measure. 
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F-22A  Advanced  Tactical  Fighter 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  completed  FOT&E  of  F-22A  Increment  3.1 
Enhanced  Global  Strike  capabilities  in  November  2011. 
FOT&E  results  demonstrated  that  the  enhanced  air-to-ground 
hardware  and  software  enable  the  F-22A  to  perform  its 
intended  offensive  counter-air  suppression  of  selected,  but  not 
all  relevant  enemy  air  defenses  in  Global  Strike  scenarios. 

•  Overall  Increment  3.1  operational  suitability  was  substantially 
improved  compared  with  the  performance  in  previous 
evaluation  periods. 

•  The  Air  Force  completed  a  formal  investigation  of  the 
aircraft  life  support  system  and  associated  onboard  oxygen 
generation  system,  and  ruled  out  contamination  as  the  root 
cause  of  hypoxia-like  incidents  that  resulted  in  the  fleet- wide 
F-22A  grounding  in  FY11.  The  Air  Force  concluded  that 
impedance/restriction  caused  by  life  support  system  elements 
were  significant  contributors  to  the  physiological  incidents  and 
is  pursuing  actions  to  resolve  the  problem  including:  removal 
of  the  aircrew  C21 A  filter  pack,  and  testing  of  a  modified 
aircrew  upper  pressure  garment.  Additionally,  the  Air  Force 

is  installing  a  back-up  emergency  oxygen  system  to  provide 
emergency  oxygen  in  the  event  of  an  environmental  control 
system  shut  down,  rapid  decompression,  or  failure  of  the 
onboard  oxygen  generator. 

•  In  conjunction  with  the  completion  of  Increment  3.1  FOT&E 
and  the  F-22A  fleet  achieving  its  “at  maturity”  100,000  fleet 
flight  hours  milestone,  DOT&E  conducted  an  analysis  of 
F-22A  progress  in  satisfying  the  original  weapon  system 
operational  suitability  requirements.  Based  on  performance 
from  IOT&E  through  Increment  3.1  FOT&E,  DOT&E 
assesses  the  mature  F-22A  weapon  system  is  operationally 
effective  and  suitable. 

System 

•  The  F-22A  is  an  air  superiority  fighter  that  combines  low 
observability  to  threat  radars,  sustained  high  speed,  and 
integrated  avionics  sensors. 

•  F-22A  low  observability  reduces  threat  capability  to  engage 
with  current  weapons. 

•  The  aircraft  maintains  supersonic  speeds  without  the  use  of  an 
afterburner. 

•  Avionics  that  fuse  information  from  the  Active  Electronically 
Scanned  Array  radar,  other  sensors,  and  datalinked  information 
for  the  pilot  enable  employment  of  medium-  and  short-range 
air-to-air  missiles,  guns,  and  air-to-ground  munitions. 

•  The  Air  Force  designed  the  F-22A  to  be  more  reliable  and 
easier  to  maintain  than  legacy  fighter  aircraft. 

•  F-22A  air-to-air  weapons  are  the  AIM-120C  radar-guided 
missile,  the  AIM-9M  infrared-guided  missile,  and  the  M61A1 
20  mm  gun. 


•  F-22A  air-to-ground  precision  strike  capability  consists  of  the 
1,000-pound  Joint  Direct  Attack  Munition  (JDAM)  and  the 
250-pound  Small  Diameter  Bomb  (SDB)  Increment  One. 

•  The  F-22A  program  delivers  capability  in  increments. 
Incremental  Enhanced  Global  Strike  modernization  efforts 
include  the  following  current  and  projected  increments: 

-  Increment  3.1  provides  enhanced  air-to-ground  mission 
capability,  to  include  geo-location  of  selected  emitters, 
electronic  attack,  air-to-ground  synthetic  aperture 
radar  mapping  and  designation  of  surface  targets,  and 
SDB  integration.  Increment  3.1  is  currently  fielded  in 
operational  F-22A  units. 

-  Increment  3.2A  is  a  software-only  upgrade  intended 
to  provide  improved  Electronic  Protection,  Link  16, 
and  Combat  Identification  capabilities  in  FY14. 

Increment  3.2A  is  a  modernization  effort  within  the 
scope  of  the  F-22  A  Advanced  Tactical  Fighter  baseline 
acquisition  program  of  record. 

-  Increment  3.2B  is  a  hardware  and  software  upgrade 
intended  to  integrate  AIM-120D  and  AIM-9X  missile 
systems,  and  provide  additional  Electronic  Protection 
enhancements  and  improved  emitter  geo-location 
capability  in  FY17.  Increment  3.2B  will  be  a  separate 
Major  Defense  Acquisition  Program  with  Milestone  B 
projected  for  December  2012. 

Mission 

A  unit  equipped  with  the  F-22  A: 

•  Provides  air  superiority  over  friendly  or  enemy  territory 

•  Defends  friendly  forces  against  fighter,  bomber,  or  cruise 
missile  attack 
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•  Escorts  friendly  air  forces  into  enemy  territory  Major  Contractor 

•  Provides  air-to-ground  capability  for  counter-air,  strategic  Lockheed  Martin  Aeronautics  Company  -  Fort  Worth,  Texas 

attack,  counter-land,  and  enemy  air  defense  suppression 

missions 


Activity 

•  The  Air  Force  conducted  F-22A  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  (TEMP) 
and  FOT&E  plans. 

•  The  Air  Force  completed  F-22A  Increment  3.1  FOT&E 
in  November  2011.  Fleet-wide  Increment  3.1  retrofits  of 
Block  30  F-22As  were  ongoing  throughout  FY12. 

•  The  Air  Force  completed  a  formal  investigation  of  the  aircraft 
life  support  system  and  associated  onboard  oxygen  generation 
system  due  to  several  unexplained  hypoxia-like  incidents  that 
occurred  throughout  FY 1 1 .  Fleet-wide  F-22A  grounding  from 
April  through  September  2011  delayed  the  planned  completion 
of  Increment  3.1  FOT&E  until  November  2011.  The  Air  Force 
investigation  ruled  out  contamination  as  the  root  cause  of  the 
incidents.  As  of  August  2012,  the  Air  Force  concluded  that 
impedance/restriction  caused  by  life  support  system  elements 
were  significant  contributors  to  the  physiological  incidents. 
Accordingly,  the  Air  Force  is  pursuing  the  following  actions 

in  the  interest  of  resolving  the  problem:  removing  the  aircrew 
C2 1 A  filter  pack,  and  testing  a  modified  aircrew  upper  pressure 
garment.  Additionally,  the  Air  Force  is  installing  a  back-up 
emergency  oxygen  system  to  provide  emergency  oxygen  in 
the  event  of  an  environmental  control  system  shut  down,  rapid 
decompression,  or  failure  of  the  onboard  oxygen  generator. 

•  The  Air  Force  conducted  planning  activities  in  support 
of  Increment  3.2A  and  3.2B  modernization  efforts. 

Increment  3. 2 A  developmental  testing  will  begin  in  FY13.  In 
conjunction  with  the  completion  of  Increment  3.1  FOT&E  and 
the  F-22A  fleet  achieving  its  “at  maturity”  100,000  fleet  flight 
hours  milestone,  DOT&E  conducted  an  analysis  of  F-22A 
progress  in  satisfying  the  original  weapon  system  operational 
suitability  requirements. 

Assessment 
Increment  3.1  FOT&E 

•  Results  of  Increment  3.1  FOT&E  testing  completed 
in  November  2011  demonstrated  that  the  enhanced 
air-to-ground  capabilities  enable  the  F-22A  to  perform  its 
intended  offensive  counter-air  suppression  of  enemy  air 
defenses  mission  in  Global  Strike  scenarios. 

-  The  F-22A  remains  capable  of  effectively  employing 
legacy  JDAM,  AIM-9M,  and  AIM-120C  weapons  as 
well  as  the  newly  incorporated  SDB. 

-  Aircrews  are  capable  of  using  the  F-22A  radar  and 
onboard  sensors  to  reliably  locate  and  designate  surface 
targets  with  sufficient  accuracy  to  effectively  employ 
both  legacy  JDAM  and  newly  incorporated  SDB 
munitions  to  suppress  selected,  but  not  all  relevant 
enemy  air  defenses. 


-  Overall  Increment  3.1  operational  suitability  was 
substantially  improved  compared  with  the  performance 
in  previous  evaluation  periods.  FOT&E  demonstrated 
a  significant  improvement  in  Mean  Time  Between 
Critical  Failure  (MTBCF)  compared  to  previous 
OT&E  periods.  Increment  3.1  FOT&E  MTBCF  was 
4.68  hours  (4.01  hours  80  percent  lower  confidence 
bound;  5.26  hours  80  percent  upper  confidence  bound) 
compared  to  the  reported  MTBCF  in  the  FY07  FOT&E 
of  1.73  hours. 

-  The  Increment  3.1  F-22A  weapons  system  exceeded 
the  operational  deployability  threshold  requirement  of 
seven  C-17  airlift  equivalents  by  one  additional  C-17. 

-  The  Increment  3.1  F-22A  weapons  system  met  both 
combat  sortie  generation  requirements  and  material 
availability  threshold  requirements. 

F-22A  System  At  Maturity  Assessment 
•  In  conjunction  with  the  completion  of  Increment  3.1 
FOT&E,  DOT&E  assessed  F-22A  progress  since  the 
completion  of  IOT&E  in  satisfying  the  original  weapon 
system’s  operational  effectiveness  and  suitability 
requirements.  Findings  include  the  following: 

-  Based  on  performance  from  IOT&E  through 
Increment  3.1  FOT&E,  DOT&E  assesses  the  mature 
F-22A  weapon  system  as  operationally  effective  in  both 
air-to-air  and  air-to-ground  mission  roles. 

-  DOT&E  compared  Increment  3.1  FOT&E  suitability 
results  with  operational  F-22A  unit  performance 
through  an  independent  model  that  simulated  the 
combat  sortie  generation  operations  of  an  operational 
F-22A  squadron.  Based  on  suitability  results  achieved 
during  Increment  3.1  FOT&E  and  results  of  the 
DOT&E  model,  DOT&E  assesses  that  the  mature 
F-22A  weapons  system  is  operationally  suitable. 

-  The  Air  Force  has  matured  maintenance  practices, 
improved  subsystem  suitability,  adjusted  manpower 
requirements,  and  modified  the  F-22A  deployment 
concept  of  operations  in  order  to  meet  F-22A  combat 
air  power  needs.  The  F-22A  weapons  system  is  capable 
of  achieving  material  availability  and  combat  sortie 
generation  Key  Performance  Parameter  thresholds. 
However,  based  on  increased  maintenance  manpower, 
equipment,  and  supplies  necessary  to  sustain  combat 
operations,  the  Air  Force  is  likely  to  continue  to  require 
eight  C-17  airlift  equivalents  to  deploy  an  F-22A 
squadron  in  support  of  global  operations. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
continues  to  address  all  previous  recommendations. 

•  FY12  Recommendation. 

1.  Commensurate  with  the  maturation  of  the  F-22A  weapons 
system  and  ongoing  F-22A  modernization  efforts,  the  Air 
Force  should  apply  past  lessons  learned  for  forthcoming 
F-22A  upgrades,  and  the  development  and  fielding  of  future 
manned  fighter  aircraft  programs.  Particular  attention 
should  be  given  to  the  challenges  of  maintaining  low 
observable  systems. 


F-22A 


251 


AIR  FORCE  PROGRAMS 


252 


HC/MC-130J 


Executive  Summary 

•  DOT&E  analysis  of  the  IOT&E  data  was  ongoing  at  the 
end  of  FY12.  DOT&E  expects  to  issue  a  Beyond  Low-Rate 
Initial  Production  (BLRIP)  report  in  2QFY13  to  inform  the 
April  2013  Full-Rate  Production  decision. 

•  Preliminary  IOT&E  results  indicate  that  the  HC/MC-130J 
provides  the  capability  to  perform  the  combat  search  and 
rescue  (CSAR)  and  Special  Operations  Forces  (SOF)  missions 
as  well  as  or  better  than  legacy  aircraft  in  many  aspects. 

•  Preliminary  IOT&E  results  indicate  that  the  HC/MC-130J 
met  the  required  availability  and  mission  capable  rates,  met 
the  maintenance  man-hour  per  flight  hour  requirement,  and 
demonstrated  a  Mean  Time  Between  Corrective  Maintenance 
significantly  better  than  the  combat  delivery  C-130J  fleet. 

•  Preliminary  IOT&E  results  indicate  that  one  aspect  of  aircraft 
survivability  may  be  degraded  relative  to  the  legacy  aircraft 
for  the  intended  concept  of  employment. 

System 

•  The  HC/MC-130J  is  a  medium-sized,  four-engine  turboprop 
tactical  transport  aircraft  with  hose  and  drogue  aerial  refueling, 
airdrop,  and  command  and  control  capabilities.  The  core 
configuration  is  based  on  the  Marine  Corps  KC-130J  refueling 
tanker  design  with  modifications  including  the  ability  to 
receive  fuel  in  flight,  a  nose-mounted  electro-optical/infrared 
sensor,  and  a  combat  systems  operator  flight  deck  station. 

•  The  HC/MC-130J  program  delivers  capability  in  increments. 
Increment  1  modifications  include  additional  countermeasure 
dispensers,  high-altitude  ramp  and  door  hydraulics,  an 
additional  (fourth)  flight  deck  crew  member  station,  an 
additional  cargo  compartment  intercom  panel,  and  cargo 
compartment  60-Hertz  electrical  outlets. 

•  The  HC-130J  will  replace  legacy  HC-130P/N  and  MC-130P 
(rescue)  aircraft;  the  MC-130J  will  replace  legacy  MC-130E/P 
aircraft.  The  Air  Force  intends  to  procure  37  HC-130Js  and  94 
MC-130Js. 


Mission 

•  Air  Combat  Command  (ACC)  uses  the  HC-130J  to  support 
the  personnel  recovery  mission  through: 

-  Aerial  and  ground  refueling  of  vertical  lift  assets  used 
during  personnel  recovery  missions 

-  Para-rescue  jumper  deployment  with  rescue-related 
equipment 

-  Infiltration/exfiltration  and  resupply  by  airdrop  or  air  land 
operations 

•  Air  Force  Special  Operations  Command  (AFSOC)  uses 
the  MC-130J  to  support  special  operations  missions 
requirements,  including: 

-  Aerial  refueling  and  forward  arming  and  refueling  point 
operations  of  SOF  rotary  and  tilt-rotor  aircraft 

-  Infiltration/exfiltration,  resupply,  or  delivery  of  SOF 
personnel  and  equipment  via  airdrop  or  landing  on 
austere,  short  runways  in  hostile  or  denied  territory 

Major  Contractor 

Lockheed  Martin  Aeronautics  Company  -  Bethesda,  Maryland 


Activity 

•  Lockheed  delivered  2  core  configuration  aircraft  and  10 
Increment  1  aircraft  to  the  Air  Force. 

•  The  46th  Operations  Group  completed  developmental  test 
and  evaluation  (DT&E)  of  the  core  configuration  aircraft 
in  September  2011  and  of  the  Increment  1  configuration, 
which  was  the  production-representative  test  article  for  the 
IOT&E,  in  February  2012.  The  Increment  1  DT&E  included 
regression  testing  of  updated  Operational  Flight  Program 
software  on  the  new  Sanders  Technology  and  Advanced 
Risk  Instruction  Set  Computer  (STAR)  IX  mission  computer 


hardware  that  is  replacing  STAR  Vll  hardware  throughout  the 
C-130J  fleet  due  to  diminishing  manufacturing  sources. 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
conducted  IOT&E  from  March  1  through  May  30,  2012,  in 
accordance  with  the  DOT&E-approved  test  plan.  At  least  two 
full  crews  each  from  ACC  and  AFSOC  flew  two  Increment  1 
aircraft  based  at  Cannon  AFB,  New  Mexico.  IOT&E  included 
approximately  253  flight  hours  over  60  test  missions,  including 
a  simulated  deployment  and  arctic  testing  at  Eielson  AFB, 
Alaska;  maritime  testing  at  Hurlburt  Field,  Florida;  and  CSAR 
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exercises  at  Nellis  AFB,  New  Mexico.  Test  scenarios  included 
air  land  and  airdrop  delivery  of  personnel  and  cargo,  aerial 
refueling  as  a  receiver  and  as  a  tanker  to  helicopters  and 
tilt-rotors,  forward  arming  and  refueling  point  operations,  and 
defensive  reactions  to  simulated  threat  engagements. 

•  The  IOT&E  also  included  scenarios  for  defensive 
engagements  with  prepositioned  and  mobile  threat  simulators 
to  assess  aircraft  survivability  equipment. 

Assessment 

•  DOT&E  analysis  of  the  IOT&E  data  was  ongoing  at  the  end  of 
FY12.  DOT&E  expects  to  issue  a  BLRIP  report  in  2QFY13  to 
inform  the  April  2013  Full-Rate  Production  decision. 

•  Preliminary  IOT&E  results  indicate  that  the  HC/MC-130J 
provides  the  capability  to  perform  the  CSAR  and  SOF 
missions  as  well  as  or  better  than  legacy  aircraft  in  many 
aspects.  The  improved  propulsion  system  enables  better 
tactical  take-off  performance  from  short,  unimproved  runways 
and  expands  the  flight  envelope  for  aerial  refueling.  The 
C-130J  enhanced  cargo  handling  system  greatly  improved 
loading,  unloading,  and  airdrop  operations  relative  to  legacy 
aircraft.  There  were  deficiencies  in  the  following  areas: 

-  Current  airdrop  procedures  result  in  very  high  workload 
and  head-down  time  for  the  pilot  monitoring  airdrop  and 
should  be  revised. 

-  Crews  commented  that  the  lack  of  a  tactical  datalink,  such 
as  Link  16,  limited  their  situational  awareness.  The  C-130J 
Block  Upgrade  7.0  was  planned  to  provide  Link  16,  but  the 
C-130J  program  has  deferred  fielding  of  Block  Upgrade  7.0 
until  Block  Upgrade  8.1  has  been  developed  and  tested. 

-  Loadmasters  have  insufficient  control  over  cargo 
compartment  lighting,  and  the  night-vision  compatible 
lighting  does  not  adequately  support  covert  operations. 

-  The  location  of  some  litter  support  strap  hangers  above 
the  centerline  overhead  avionics  equipment  rack  hinders 
configuration  of  the  cargo  compartment  for  medevac 
operations. 


-  Lack  of  several  specialized  features  for  search  and  rescue 
missions  relative  to  legacy  aircraft  (flare  launcher  tubes, 
large  forward  scanner  windows,  additional  oxygen 
regulators  and  intercom  panels)  may  require  ACC  or  the 
Air  National  Guard  to  make  modifications  to  the  aircraft 
after  delivery. 

-  The  intercom  system  does  not  transmit  system  tones 
(diagnostic  or  defensive  system  alerts)  to  all  intercom 
panels  in  the  cargo  compartment,  limiting  loadmasters’ 
situational  awareness. 

•  Preliminary  IOT&E  results  indicate  that  the  HC/MC-130J 
met  the  required  availability  and  mission-capable  rates,  met 
the  maintenance  man-hour  per  flight  hour  requirement,  and 
demonstrated  a  Mean  Time  Between  Corrective  Maintenance 
significantly  better  than  the  combat  delivery  C-130J  fleet. 

•  Preliminary  IOT&E  results  indicate  that  one  aspect  of  aircraft 
survivability  may  be  degraded  relative  to  the  legacy  aircraft 
for  the  intended  concept  of  employment.  Although  the  aircraft 
survivability  equipment  installed  on  the  HC/MC-130J  has 
been  tested  and  employed  on  other  C-130J  aircraft,  it  may 
exhibit  shortfalls  under  the  new  mission-specific  concepts  of 
employment. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Address  cargo  compartment  lighting  control  by 
loadmasters,  including  AFSOC’s  need  for  blacked-out  cargo 
compartment  lighting  in  covert  operations. 

2.  Develop  mitigation  plans  for  the  deferred  release  of  Block 
Upgrade  7.0,  including  the  need  for  a  tactical  datalink. 

3.  Develop  plan  to  integrate  improvements  of  aircraft 
survivability  equipment  in  future  increments. 
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Integrated  Strategic  Planning  and  Analysis  Network 

(ISPAN) 


Executive  Summary 

•  The  overall  Integrated  Strategic  Planning  and  Analysis 
Network  (ISPAN)  program  consists  of  a  system-of-systems 
approach  that  spans  multiple  security  enclaves  for  strategic 
and  operational-level  planning  and  leadership  decision 
making.  ISPAN  Increment  2  employed  an  agile  acquisition 
approach  that  used  two  spirals  to  deliver  this  capability. 

•  U.S.  Strategic  Command  (USSTRATCOM)  and  the  Air  Force 
Operational  Test  and  Evaluation  Center  (AFOTEC)  conducted 
an  ISPAN  Increment  2,  Spiral  2,  IOT&E  in  accordance  with 
the  DOT&E-approved  test  plan  in  July  2012. 

•  The  IOT&E  confirmed  that  the  problems  discovered  during 
ISPAN  Increment  2,  Spiral  1,  Integrated  Test  and  Evaluation 
(IT&E),  which  USSTRATCOM  and  AFOTEC  conducted  in 
December  2011,  were  resolved. 

•  DOT&E  assessed  ISPAN  Increment  2  as  effective  but 
not  suitable  due  to  system  limitations  involving  software 
deficiencies,  problematic  configuration  management 
processes,  potential  scalability  deficiencies,  and  shortfalls  in 
usability,  timeliness,  training,  documentation,  and  maintenance 
of  system  interoperability. 

•  Web  services  provide  interoperability  functionality  that  can  be 
unavailable  when  data  structures  are  changed,  interfaces  are 
upgraded,  or  other  events  occur  that  affect  how  these  services 
are  implemented.  The  Program  Office  should  negotiate 
service-level  agreements  between  web  interface  partners  to 
ensure  that  web  services  remain  available  and  facilitate  critical 
net-centric  data  exchanges. 

•  AFOTEC  discovered  a  number  of  Information  Assurance  (IA) 
vulnerabilities  during  the  IOT&E.  The  Program  Office  must 
promptly  mitigate  the  two  Category  1 1A  vulnerabilities  found 
during  IOT&E  and  develop  a  plan  of  action  and  milestones  to 
address  the  Category  11  and  III  findings. 

•  The  Designated  Accrediting  Authority  (DAA)  reviewed  the 
program  manager-provided  mitigation  plan  and  issued  an 
Authority  to  Operate  on  December  14,  2011.  The  Secretary 
of  the  Air  Force  granted  a  Full-Deployment  decision  in 
October  2012. 

System 

•  The  overall  ISPAN  program  consists  of  a  system-of-systems 
approach  that  spans  multiple  security  enclaves  for  strategic 
and  operational-level  planning  and  leadership  decision 
making. 

•  The  ISPAN  system  is  composed  of  two  elements,  which  are 
developed  in  increments: 

-  The  Global  Adaptive  Planning  Collaborative 

Information  Environment  (GAP  CIE)  that  manages 
strategy-to-execution  planning  across  all  USSTRATCOM 


Operation  Planning 


Common  Data  Schemas, 
Vocabulary  and  Service 
Level  Agreements 

Net-enabled 
Enterprise  Services 

Federated  & 
Synchronized  Data  at 
Multiple  Nodes 

CCDR  -  Combatant  Commander  SCI  -  Special  Compartmented  Information 

COA  -  Course  of  Action  STO/SAP  -  Special  Technical  Operations/Special  Access  Programs 

JTF/JSC  -  Joint  Task  Force/Joint  Force  Commander  TS  -  Top  Secret 
S- Secret  U  -  Unclassified 

mission  areas,  which  is  being  developed  as  part  of 
Increments  2  and  3. 

-  The  Mission  Planning  and  Analysis  System  (MPAS)  that 
develops  Joint  Staff  Level  I  through  Level  IV  nuclear  and 
conventional  force  application  plans  that  support  national 
and  theater  requirements,  which  will  be  modernized  as 
part  of  Increment  4.  Increment  4  is  scheduled  to  begin 
development  in  FY14. 

•  Prior  to  Increment  2,  the  initial  capabilities  of  GAP  CIE  and 
MPAS  were  fielded  in  ISPAN  Block  1.  After  the  fielding  of 
ISPAN  Block  1,  the  acquisition  nomenclature  was  changed 
from  a  Block  to  Increment  designation. 

•  ISPAN  is  employing  an  agile  acquisition  methodology,  which 
delivers  system  capabilities  in  increments  depending  on  user 
requirements.  Increment  2  IOT&E  consisted  of  approximately 
40  percent  of  the  planned  system  capabilities.  The  developers 
intend  to  deliver  the  remaining  60  percent  in  Increment  3, 
which  is  presently  unfunded.  USSTRATCOM  intends  to 
begin  developing  ISPAN  MPAS  capabilities  in  Increment  4. 

•  GAP  CIE  provides  a  web-enabled,  net-centric  collaborative 
environment  for  a  contingency  and  crisis  action  planning 
(CAP)  system  at  the  Combatant  Command  (CCMD)  and 
strategic  level.  The  capability  allows  users  from  multiple 
CCMD  staffs,  subordinate  commands,  as  well  as  other 
agencies,  to  collaborate  online  while  providing  planning  and 
analyses  to  senior  decision-makers. 

•  MPAS  is  intended  to  provide  dedicated  planning  and  analysis 
for  all  U.S.  strategic  nuclear  forces,  in  addition  to  planning 
and  analysis  to  create  plans  for  specified  theater  and  strategic 
conventional  forces. 
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•  The  ISPAN  Increment  2  provides  an  Adaptive  Planning 
capability  to  CCMDs  and  subordinate  planning  partners, 
while  decreasing  contingency  and  CAP  timelines  to  meet  the 
DoD-approved  Adaptive  Planning  and  execution  process,  as 
well  as  time  sensitive  Global  Strike  and  Prompt  Global  Strike 
planning  requirements.  The  ISPAN  Increment  2  is  intended  to 
improve  information  access  through  net-centric  means  for  time 
critical  event  management  support  to  senior  decision-makers, 
and  deliver  the  ISPAN  GAP  CIE  capability  as  a  service  to 

the  Enterprise,  by  exposing  critical  ISPAN  information  as 
Net-Centric  Enterprise  Services  registered  web  services. 

Mission 

•  USSTRATCOM  uses  ISPAN  to  perform  deliberate  and 
adaptive,  strategic,  nuclear,  and  non-nuclear  planning  and 


analysis.  This  includes  developing  the  national  deterrence  war 
plans  offering  both  nuclear  and  non-nuclear  weapon  options 
using  the  MPAS. 

•  The  CCMDs,  subordinate  staffs,  and  other  national  agencies 
use  GAP  CIE  for  collaborative  mission  planning  and  analysis, 
course  of  action  development,  and  commander’s  decision 
briefing  preparation  in  support  of  CAP  scenarios  and  time 
critical  decisions  regarding  force  employment. 

Major  Contractors 

•  Lockheed  Martin  -  Papillion,  Nebraska 

•  BAE  Systems  -  Bellevue,  Nebraska 

•  Northrop  Grumman  -Bellevue,  Nebraska 

•  SAIC  -  San  Diego,  California 


Activity 

•  USSTRATCOM  and  AFOTEC  conducted  an  ISPAN 
Increment  2,  Spiral  1  IT&E  at  USSTRATCOM,  Offutt  AFB, 
Nebraska,  and  the  Combined  Air  Operations  Center,  Barksdale 
AFB,  Louisiana,  in  December  2011.  AFOTEC  did  not 
conduct  the  IT&E  in  accordance  with  a  DOT&E-approved  test 
plan  and  collected  most  data  from  a  single  Integrated  Mission 
Area  Training  exercise,  rather  than  the  three  CAP  events 
planned.  In  addition,  IA  testing  was  not  conducted. 

•  In  July  2012,  USSTRATCOM  and  AFOTEC  conducted  an 
ISPAN  Increment  2,  Spiral  2  IOT&E  at  USSTRATCOM, 
Offutt  AFB,  Nebraska;  the  Combined  Air  Operations  Center, 
Barksdale  AFB,  Louisiana;  and  Headquarters,  U.S.  Southern 
Command,  Miami,  Florida.  AFOTEC  conducted  the  IOT&E 
in  accordance  with  the  DOT&E-approved  test  plan. 

•  The  Secretary  of  the  Air  Force  granted  a  Full-Deployment 
decision  in  October  2012. 

Assessment 

•  CCMD  staffs  can  effectively  use  ISPAN  Increment  2  to 
provide  improvements  to  the  planning  process  to  achieve 
operational  outcomes.  Users  were  able  to  accomplish  all 
necessary  planning  functions  in  a  collaborative  manner, 
including  users  at  two  different  locations. 

•  During  IOT&E,  the  operational  test  team  was  able  to  complete 
the  principal  functions  of  the  ISPAN  Increment  2  system:  the 
development  of  Courses  of  Action  (COAs)  and  the  integration 
of  multiple  COAs  for  the  commander’s  approval. 

•  ISPAN  Increment  2  was  not  suitable.  The  system  limitations 
involved  software  deficiencies,  problematic  configuration 
management  processes,  potential  scalability  deficiencies,  and 
shortfalls  in  usability,  timeliness,  training,  documentation,  and 
maintenance  of  system  interoperability.  During  operational 
testing,  ISPAN  Increment  2  satisfied  the  requirements  for 
reliability,  availability,  and  maintainability  (RAM).  DOT&E 
assessed  the  areas  of  supportability,  interoperability,  transfer 
of  operations,  and  RAM  as  operationally  suitable.  DOT&E 


noted  that  some  of  these  limitations  are  requirements 
for  Increment  3 ;  however,  since  Increment  3  is  presently 
unfunded,  the  system  was  assessed  to  the  Capability 
Development  Document  requirement. 

•  In  DOT&E’s  October  20 12  IOT&E  report  on  ISPAN,  DOT&E 
assessed  configuration  management  as  not  suitable  due  to 
AFOTEC’s  discovery  of  several  configuration  management 
shortcomings  during  developmental  testing  and  IOT&E  that 
affected  the  ISPAN  mission.  The  most  critical  incident  was 
when  a  48-minute  outage  occurred  during  IOT&E  because  a 
maintenance  contractor  working  on  a  switch  within  the  system 
accidentally  disconnected  the  system,  which  resulted  in  a 
critical  failure  of  the  system.  Since  the  completion  of  IOT&E, 
the  Program  Office  has  put  in  place  updated  procedures  to 
tighten  configuration  control  of  the  system  to  prohibit  these 
issues  from  reoccurring. 

•  ISPAN  demonstrated  the  capability  to  support  370  concurrent 
users  including  simulated  users  performing  ISPAN 
representative  tasks  with  a  server  response  time  of  5  seconds 
or  less.  The  concurrent  user  capacity  is  acceptable  in  view  of 
the  near  term  user  levels  of  about  140  concurrent  users  during 
CCMD  events.  However,  when  the  simulation  was  extended 
to  a  larger  number  of  users,  the  server  response  time  increased 
significantly.  These  results  indicate  a  scalability  problem,  but 
it  is  unclear  how  many  concurrent  users  woidd  degrade  the 
user  experience  below  an  acceptable  level  of  responsiveness. 

•  Web  services  provide  interoperability  functionality  that  can  be 
unavailable  when  data  structures  are  changed,  interfaces  are 
upgraded,  or  other  events  occur  that  affect  how  these  services 
are  implemented. 

•  User  feedback  indicates  that  the  test  schedule  did  not  provide 
adequate  training  time/exposure  and  documentation  to  all 
IOT&E  participants;  therefore,  user  training  is  not  suitable. 
Some  U.S.  Southern  Command  users  were  unable  to  attend  the 
training  but  those  who  did  attend  found  it  inadequate. 
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•  AFOTEC  discovered  a  number  of  1A  vulnerabilities  during 
the  IOT&E.  DOT&E  recommended  that  the  Program  Office 
correct  or  mitigate  the  IA  vulnerabilities  to  the  satisfaction  of 
the  DAA  prior  to  fielding.  The  Program  Office  continues  to 
mitigate  IA  vulnerabilities. 

Recommendations 

•  Status  of  Previous  Recommendations.  USSTRATCOM  and 
the  ISPAN  Program  Office  have  effectively  addressed  all 
previous  recommendations. 

•  FY12  Recommendations.  The  following  recommendations  are 
from  DOT&E’s  October  2012  IOT&E  report. 

1 .  The  Program  Office  must  tightly  control  configuration 
management  of  the  ISPAN  baseline  to  prevent  incidents  and 
outages  of  the  system. 

2.  The  Program  Office  needs  to  continue  to  diagnose  and 
address  system  latency  problems  during  operations  and 
sustainment.  As  the  ISPAN  user  community  increases 
above  the  Increment  2  threshold,  poor  response  times  will 
affect  the  acceptance  of  this  system. 

3.  The  Program  Office  must  implement  a  robust  training 
program  to  include  classroom  tutorial  instruction  and 


on-the-job  training  that  is  tailored  to  the  need  of  each 
CCMD.  Training  should  be  provided  to  users  with  regards 
to  what  documentation  is  available  and  how  to  access  it. 

4.  The  Program  Office  should  negotiate  service -level 
agreements  between  web  interface  partners  to  ensure 
that  web  services  remain  available  and  facilitate  critical 
net-centric  data  exchanges.  Web  services  can  be 
unavailable  when  data  structures  are  changed,  interfaces 
are  upgraded,  or  other  events  occur  that  affect  how  these 
services  are  implemented. 

5.  The  Program  Office  should  support  the  service-level 
agreements  with  automated  tests  of  availability  of  the  web 
services. 

6.  The  Program  Office  must  promptly  mitigate  the  two 
Category  I  IA  vulnerabilities  found  during  IOT&E  and 
develop  a  plan  of  action  and  milestones  to  address  the 
Category  II  and  Ill  findings. 

7.  The  DAA  should  conduct  an  IA  assessment  for  ISPAN  in 
the  Joint  Worldwide  Intelligence  Communications  System 
environment  to  ensure  that  no  Category  1  vulnerabilities 
exist,  all  other  vulnerabilities  are  identified,  and  plans  for 
correction  of  deficiencies  are  created. 
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Joint  Air-to-Surface  Standoff  Missile  (JASSM) 


Executive  Summary 

•  The  Air  Force  executed  16  AGM-158B  Joint  Air-to-Surface 
Standoff  Missile  -  Extended  Range  (JASSM-ER)  live  fire 
shots  in  FY11  and  12,  completing  the  IOT&E  effort.  All  16 
prosecuted  their  targets  successfully. 

•  The  Air  Force,  in  conjunction  with  the  major  contractor 
Lockheed  Martin,  is  in  the  process  of  engineering 
development  of  the  FMU-162/B  Electronic  Safe  and  Arm  Fuze 
(ESAF)  as  a  replacement  for  the  mechanical  fuzes  currently 
used  in  JASSM. 

•  The  Air  Force  completed  AGM- 158 A  Lot  6  Reliability 
Assessment  Program  (RAP)  testing  with  three  shots  in  FY12, 
two  of  which  were  successful,  while  one  was  terminated  early 
to  maintain  range  safety.  There  were  three  successful  Lot  6 
RAP  shots  in  FY1 1  and  one  failed  shot  in  FY10. 

•  The  Air  Force  should  continue  to  conduct  RAP  on  Lot  8  and 
later  via  the  Weapons  System  Evaluation  Program  (WSEP). 

System 

•  The  baseline  AGM-158A  JASSM  is  a  stealthy  cruise  missile 
that  flies  a  preplanned  route  from  launch  to  a  target,  using  GPS 
guidance  and  an  internal  navigation  system.  JASSM: 

-  Lias  a  1,000-pound  penetrating  warhead 

-  Has  an  imaging  infrared  seeker  that  can  be  used  for  greater 
accuracy  and  precision;  the  seeker  uses  image  templates 
prepared  by  a  rear  echelon  intelligence  unit 

-  Can  be  launched  by  B-l,  B-2,  B-52,  F-15E,  and  F-16 
aircraft 

-  Includes  a  container  that  protects  the  weapon  in  storage 
and  aids  ground  crews  in  moving,  loading,  and  checking 
the  missile 

-  Uses  the  same  Air  Force  mission  planning  systems  used 
for  aircraft  and  other  weapons 

•  AGM-158B  JASSM-ER  is  intended  to  fly  longer  ranges 
using  a  more  efficient  engine,  larger  capacity  fuel  tanks,  and 
other  modified  components,  all  within  the  existing  fuselage. 
JASSM-ER  is  currently  integrated  on  the  B-l. 

•  FMU-162/B  ESAF  takes  advantage  of  advances  in  fuze 
technology  and  is  intended  to  be  a  more  reliable  fuze  with 


the  same  capabilities  as  the  baseline  mechanical  fuze.  The 
FMU-162/B  ESAF  would  be  used  in  JASSM  baseline  and  ER 
variants. 

Mission 

•  Operational  units  equipped  with  JASSM  will  employ  the 
weapon  from  multiple  aircraft  platforms  against  high-value  or 
highly-defended  targets  from  outside  the  lethal  range  of  many 
threats.  Units  equipped  with  JASSM  will: 

-  Destroy  targets  with  minimal  risk  to  flight  crews  and 
support  air  dominance  in  the  theater 

-  Strike  a  variety  of  targets  greater  than  200  miles  away 

-  Execute  missions  using  automated  preplanning  or  manual 
pre-launch  retargeting  planning 

-  Attack  a  wide  range  of  targets  including  soft,  medium,  and 
very  hard  (not  deeply  buried)  targets 

•  Units  with  JASSM-ER  will  support  the  same  missions  as 
JASSM  using  a  missile  with  a  range  more  than  twice  as  long. 

Major  Contractor 

Lockheed  Martin,  Missile  and  Fire  Control  -  Orlando,  Florida 


Activity 

AGM-158A  JASSM 


•  The  Air  Force  certified  JASSM  for  carriage  and 
employment  on  the  F-15E  Strike  Eagle.  F-15E  is  the 
first  JASSM-capable  aircraft  incorporating  the  Universal 
Armament  Interface,  a  plug-and-play  software  that  shortens 
the  time  required  to  integrate  new  weapons  onto  aircraft. 


There  were  three  Lot  6  RAP  test  flight  shots  in  FY12,  two 
of  which  were  successful,  while  one  was  terminated  early  to 
maintain  range  safety.  Combined  with  the  three  successful 
FY11  shots  and  the  single  failed  Lot  6  RAP  test  shot 
conducted  early  in  FY10,  this  constitutes  the  completion 
of  the  Lot  6  RAP  testing.  The  Lot  6  RAP  tests  used  an 
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equal  mix  of  re-worked  early  production  Lot  6  missiles  and 
current  configuration  Lot  6  missiles,  ensuring  adequate  and 
equal  testing  of  both  variants. 

AGM-158B  JASSM-ER 

•  The  Air  Force  executed  16  JASSM-ER  live  fire  shots  in 
FY11  and  12.  All  16  JASSM-ER  missiles  engaged  and 
destroyed  their  targets  at  both  nominal  and  maximum 
JASSM-ER  ranges. 

FMU-162/B  JASSM  ESAF 

•  The  FMU-162/B  ESAF  currently  is  funded  by  the  major 
JASSM  contractor.  The  Air  Force,  in  conjunction  with 
the  major  contractor,  is  in  the  process  of  re-design  and 
engineering  development  of  the  FMU-162/B  ESAF  as  an 
option  to  replace  the  existing  mechanical  fuze  and  thereby 
increase  JASSM  and  JASSM-ER  reliability. 

•  The  Air  Force  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

Assessment 

AGM-158A  JASSM 

•  An  April  2011  DOT&E  memorandum  stated  that  the  Air 
Force  needed  to  execute  a  minimum  of  6  RAP  flights  to 
demonstrate  an  80  percent  confidence  level  that  Lot  6 

is  more  reliable  than  Lot  5.  Currently,  the  RAP  shots 
following  the  first  failure  have  been  successful.  Omitting 
the  one  flight  that  range-safety  terminated  due  to  crew  error, 
RAP  testing  has  demonstrated  improved  reliability  over 
Lot  5. 

•  Despite  improvements  in  workmanship  and  production 
processes,  there  is  still  a  need  to  evaluate  the  inherent 


reliability  of  production  lot  missiles  (through  Lot  8,  at 
a  minimum)  to  ensure  that  the  reliability  growth  plan  is 
successful.  The  Air  Force  should  accomplish  Lot  8  and 
later  RAP  testing  via  the  WSEP. 

AGM-158B  JASSM-ER 

•  The  16  JASSM-ER  shots  indicate  that  the  JASSM-ER 
meets  requirements.  These  16  tests,  in  combination  with 

5  approved  production-representative  integrated  test  shots, 
successfully  conclude  the  IOT&E.  There  was  a  single 
failure  (IT-8)  in  21  test  events. 

FMU-162/B  JASSM  ESAF 

•  The  FMU-162/B  ESAF  program  has  the  potential  to 
increase  the  overall  reliability  of  all  JASSM  variants.  The 
FMU-162/B  ESAF  program  would  replace  the  current 
electro-mechanical  fuze,  which  relies  on  moving  parts 
prone  to  reliability  failures.  In  addition,  the  FMU-162/B 
ESAF  has  more  built-in  test  capability  than  the  current 
electro-mechanical  FMU-156/B  ESAF. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 

continued  to  characterize  the  reliability  of  baseline  missile 

production  lots  with  the  completion  of  the  Lot  6  RAP. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  In  conjunction  with  the  contractor,  continue  the 
development  and  evaluation  of  the  FMU-162/B  ESAF. 

2.  Continue  RAP  for  Lot  8  and  later  via  the  WSEP. 
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Executive  Summary 

•  The  KC-46A  contract  is  firm-fixed-price  with  incentives 
for  Engineering  and  Manufacturing  Development.  The 
Air  Force  awarded  the  contract  to  the  Boeing  Company  in 
February  2011. 

•  The  most  recent  DOT&E  review  of  the  post-Milestone  B  draft 
Test  and  Evaluation  Master  Plan  (TEMP)  indicates  that  the  Air 
Force  has  made  progress  addressing  test  execution  problems 
but  further  work  is  needed. 

•  The  ALR-69A  Radar  Warning  Receiver  (RWR)  has 
effectiveness  shortfalls  that  require  resolution  prior  to 
integration  on  the  KC-46A. 

System 

•  The  KC-46A  aerial  refueling  aircraft  is  the  first  increment 
(179)  of  replacement  tankers  for  the  Air  Force’s  fleet  of 
KC-135  tankers  (more  than  400).  The  KC-46A  design  uses 
a  modified  Boeing  767-200ER  commercial  airframe  with 
numerous  military  and  technological  upgrades,  such  as  the 
fly-by-wire  refueling  boom,  the  remote  air  refueling  operator’s 
station,  additional  fuel  tanks  in  the  body,  and  defensive 
systems.  The  KC-46A  is  intended  to  provide  boom  (pictured) 
and  probe-drogue  refueling  capabilities.  The  Air  Force  intends 
to  equip  the  KC-46A  with  an  air-refueling  receptacle  so  that 

it  can  also  receive  fuel  from  other  tankers,  including  legacy 
aircraft. 

•  The  KC-46A  is  designed  to  have  significant  palletized 
cargo  and  aeromedical  capacities,  chemical/biological/ 
radiological/ nuclear  survivability,  and  the  ability  to  host 
communications  gateway  payloads. 

•  Survivability  enhancement  features  are  incorporated  into  the 
KC-46  design.  Susceptibility  is  reduced  with  an  Aircraft 
Survivability  Equipment  suite  consisting  of  Large  Aircraft 
Infrared  Countermeasures  (LA1RCM),  the  ALR-69A  RWR, 
and  a  Tactical  Situational  Awareness  System.  The  suite  is 


intended  to  compile  threat  information  from  the  RWR  and 
other  on  and  off-board  sources  and  prompt  the  crew  with 
an  automatic  re-routing  suggestion  in  the  event  of  a  threat. 
Vulnerability  is  reduced  through  the  addition  of  fuel  tank 
inerting  and  integral  armor  to  provide  some  protection  to  the 
crew  and  critical  systems. 

Mission 

Commanders  will  use  units  equipped  with  the  KC-46  A  to: 

•  Perform  air  refueling  to  accomplish  six  primary  missions: 
nuclear  operations  support,  global  strike,  air  bridge  support, 
aircraft  deployment,  theater  support,  and  special  operations 
support.  Secondary  missions  will  include  airlift,  aeromedical 
evacuation,  emergency  aerial  refueling,  air  sampling,  and 
support  of  combat  search  and  rescue. 

•  Operate  in  day/night  and  adverse  weather  conditions  over  vast 
distances  to  support  U.S.,  joint,  allied,  and  coalition  forces. 

Major  Contractor 

The  Boeing  Company,  Commercial  Aircraft  in  conjunction  with 

Defense,  Space  &  Security  -  Seattle,  Washington 


Activity 

•  The  KC-46  A  Integrated  Test  Team  has  met  quarterly  since 
April  2011. 

•  Developmental,  operational,  and  Federal  Aviation 
Administration  test  planning  are  ongoing,  with  the  most 
recent  emphasis  on  developing  an  integrated  test  plan  and  a 
post-Milestone  B  TEMR 

•  The  Air  Force  submitted  an  operational  assessment  1  plan. 
This  assessment  does  not  include  flying  since  delivery  of  the 
first  aircraft  is  not  until  June  2014. 


•  Survivability  assessment  planning  is  ongoing,  integrating 
developmental,  operational,  and  live  fire  testing  and  analysis. 
Susceptibility  and  vulnerability  stakeholders  are  coordinating 
hardware-in-the-loop  LA1RCM  test  planning  to  confirm 
system  effectiveness  using  a  modified  767  airframe  model 
and  to  develop  hit  point  distributions  for  the  vulnerability 
assessment. 

•  The  Air  Force  acquired  two  Boeing  767-200  assets  for  live 
fire  testing  at  the  Weapons  Survivability  Lab  at  China  Lake, 
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California.  The  Air  Force  developed  a  comprehensive  live  fire 
test  plan  and  began  the  ballistic  test  program  in  FY12  with  two 
test  series: 

-  A  replica  wing  dry  bay  fire  test  series,  completed  in  FY12 
at  Wright-Patterson  AFB,  Ohio,  as  part  of  a  risk  reduction 
effort  to  characterize  various  threats  against  KC-46 
production  materials 

-  Wing  dry  bay  fire  testing  on  a  767-200  production  asset, 
which  began  on  schedule  in  3QFY12. 

Assessment 

•  The  most  recent  DOT&E  review  of  the  post-Milestone  B 
draft  TEMP  indicates  that  the  Air  Force  has  made  progress 
addressing  test  execution  problems  but  further  work  is  needed. 

•  The  draft  TEMP  now  allocates  5.5  months  and  750  hours  for 
operational  testing,  which  is  adequate  for  an  effectiveness 
evaluation.  To  assess  suitability,  a  statistically  significant 
evaluation  (at  a  76  percent  confidence)  requires  1,250  flight 
hours;  IOT&E  may  have  to  be  extended  to  generate  the 
required  1,250  flight  hours  if  operationally  representative  field 
flight  data  are  not  available  from  other  sources. 

•  The  planned  test  program  includes  the  following  shortfalls  that 
have  been  partially  addressed  but  require  complete  resolution 
to  gain  DOT&E  approval: 

-  Concurrent  activities  and  planned  flying  hours  for  the 
Engineering,  Manufacturing,  and  Development  program 
place  a  high  demand  on  limited  aircraft  and  simulator 
resources. 

-  The  planned  effectiveness  for  military  flight  testing 
substantially  exceeds  the  relevant  historical  experience. 

The  planned  75  percent  test  effectiveness  rate  for 
military  test  points  is  not  consistent  with  experience  for 
flight  testing  of  military  aircraft.  An  average  historical 
test  effectiveness  rate  of  55  percent  combined  with 

30  flight-hours-per-aircraft-per-month  would  extend 
the  current  1 6-month  schedule  for  military  testing  by 
approximately  4-7  months,  a  best-case  estimate. 

-  The  Test  Program  Schedule  in  the  draft  TEMP  has 
insufficient  calendar  time  allotted  for  correction  of 
discrepancies  and/or  deficiencies  discovered  during 
developmental  testing  prior  to  the  planned  start  of 
operational  testing. 


-  The  draft  TEMP  allocates  insufficient  resources  (time  and 
aircraft)  for  the  initial  training  of  aircrews  and  maintainers 
and  technical  order  verification.  Additional  training  time 
between  the  end  of  developmental  test  and  the  start  of 
IOT&E  is  required. 

•  The  susceptibility  assessment  will  focus  on  LAIRCM 
effectiveness  and  situational  awareness  provided  by  the 
RWR  and  Tactical  Situational  Awareness  System.  Since 
the  majority  of  data  for  the  evaluation  of  susceptibility  will 
come  from  contractor  and  developmental  testing,  and  this 
testing  is  governed  by  the  firm-fixed-price  contract,  increases 
in  the  scope  of  testing  identified  by  DOT&E  may  require 
additional  funding.  The  building  block  design  of  experiments 
approach  taken  for  dry  bay  fire  testing  will  provide  a  method 
for  wing  leading  and  trailing  edge  damage  prediction  that 
should  be  useful  for  reducing  uncertainty  in  the  model-based 
vulnerability  assessment. 

•  The  ALR-69A  RWR  was  selected  as  Contractor  Furnished 
Equipment  by  Boeing;  however,  integration  and  performance 
on  the  KC-46  A  are  high  risk.  DOT&E  recently  completed 
an  assessment  of  the  ALR-69A  RWR  on  the  C-130H1  and 
assessed  it  as  not  effective,  but  suitable,  in  a  separate  classified 
report  dated  October  22,  2012.  Not  only  do  these  effectiveness 
problems  require  correction,  but  the  system  is  required  to 
improve  its  geo-location  capabilities  as  compared  to  the 
demonstrated  C-130J  capability. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has 
addressed  some  of  the  FY 1 1  recommendations  to  submit  a 
TEMP  incorporating  realistic  assumptions;  however,  additional 
work  is  needed. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Submit  a  TEMP  with  a  realistic  schedule  mitigating  the 
above  mentioned  shortfalls. 

2.  Correct  ALR-69A  shortfalls  prior  to  integration  on  the 
KC-46A. 

3.  Plan  to  begin  IOT&E  at  least  six  months  later  than  the 
current  draft  TEMP  indicates  to  allow  for  completion  of 
developmental  test  and  initial  training. 
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Massive  Ordnance  Penetrator  (MOP) 


Executive  Summary 

•  DOT&E  published  a  classified  Massive  Ordnance  Penetrator 
(MOP)  Early  Fielding  report  in  April  2012  that  summarized 
testing  during  FY08  through  FY11,  including  five  B-2  Quick 
Reaction  Capability  flight  tests. 

•  The  Air  Force  executed  two  MOP  sled  tests  at  the  Holloman 
AFB  High-Speed  Test  Track  during  June  and  August  2012 
to  confirm  the  successful  re-design  of  certain  aspects  of  the 
weapon  system. 

•  The  Air  Force,  between  June  and  October  2012,  successfully 
completed  five  additional  weapon  drops  from  the  B-2  aircraft 
on  threat-representative  targets.  The  tests,  conducted  at  the 
White  Sands  Missile  Range,  New  Mexico,  further  defined 
weapon  behavior  against  the  target  sets. 

System 

•  The  MOP,  GBU-57A/B,  is  a  large  penetrating  weapon  with 
the  ability  to  attack  deeply  buried  and  hardened  bunkers 
and  tunnels.  The  warhead  case  is  made  from  a  special  high 
performance  steel  alloy  and  its  design  allows  for  a  large 
explosive  payload  while  maintaining  the  integrity  of  the 
penetrator  case  during  impact. 

•  The  B-2  Spirit  is  the  only  aircraft  in  the  Air  Force  programmed 
to  employ  the  MOP. 

•  The  MOP  is  a  GPS-guided  weapon  designed  to  reach  and 
destroy  targets  located  in  well  protected  facilities.  The  MOP 
is  more  powerful  than  its  predecessors,  the  BLU-109  and 
GBU-28. 


Mission 

Combatant  Commanders  use  MOP  to  conduct  pre-planned,  day 
or  night  attacks  against  defended  point  targets  vulnerable  to  blast 
and  fragmentation  effects  and  requiring  significant  penetration, 
such  as  hardened  and  deeply-buried  facilities. 

Major  Contractor 

The  Boeing  Company,  Defense,  Space  &  Security  -  St.  Louis, 
Missouri 


Activity 

•  The  Air  Force,  using  the  Holloman  AFB  High  Speed  Test 
Track  in  New  Mexico,  conducted  two  sled  tests  during 
the  June  to  August  2012  period,  to  confirm  the  successful 
re-design  of  a  critical  part  of  the  weapon  system. 

•  The  Air  Force  executed  five  weapon  drops  at  White  Sands 
Missile  Range,  New  Mexico,  between  June  and  October 
2012,  on  threat-representative  targets  to  further  evaluate 
weapon  system  performance  and  to  provide  additional 
confirmation  of  the  weapon  re-design.  During  the  tests,  a 
B-2  conducted  five  drops:  three  with  live  warheads,  and  two 
with  inert  warheads.  Telemetry  data  and  visual  observations 
indicate  that  all  five  weapon  drops  effectively  prosecuted 
the  targets. 


Assessment 

•  In  the  April  2012  Early  Fielding  report,  DOT&E  concluded 
that  the  MOP  is  capable  of  effectively  prosecuting  selected 
hardened,  deeply  buried  targets.  All  recommendations  in  the 
Early  Fielding  report  have  been  addressed  by  the  Air  Force. 

•  The  sled  test  results  and  the  additional  weapon  drops  indicate 
that  the  weapon  re-design  is  adequate  for  the  successful 
prosecution  of  all  of  the  elements  of  the  currently  defined 
target  set. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY12  Recommendations.  None. 


MOP  263 


AIR  FORCE  PROGRAMS 


264 


Miniature  Air-Launched  Decoy  (MALD) 
and  MALD-Jammer  (MALD-J) 


Executive  Summary 

•  DOT&E’s  April  2011  IOT&E  report  assessed  the  Miniature 
Air-Launched  Decoy  (MALD)  as  operationally  effective  for 
combat,  but  not  operationally  suitable  due  to  poor  materiel 
reliability,  in  July  2011,  the  Air  Force  identified  a  fault 
with  the  missile’s  radio  frequency  connector  that  caused  it 
to  separate  from  the  missile  during  long-endurance  carriage 
flights.  The  Air  Force  has  repaired  the  fault  and  conducted 
further  reliability  testing;  however,  MALD  operational 
reliability  of  78  percent  remains  below  the  93  percent 
threshold  requirement. 

•  The  Air  Force  will  no  longer  procure  any  MALDs,  as  the 
Program  Office  converted  the  MALD  procurement  line  to 
MALD-Jammer  (MALD-J). 

•  The  Air  Force  demonstrated  corrective  actions  for 
long-endurance  carriage  time  failures  with  an  additional 
MALD-J  Engineering,  Manufacturing,  and  Development 
(EMD)  test  mission  in  March  2012. 

•  The  Air  Force  launched  14  MALD/MALD-J  shots  during 
FY12  without  failure. 

•  Limited  accessibility  to  test  ranges,  unavailability  of  threat 
systems,  and  delays  in  processing  and  evaluating  data  have 
hampered  MALD  and  MALD-J  testing.  The  Air  Force  needs 
to  allocate  sufficient  range  time  for  testing  and  reduce  data 
processing  turnaround  times. 

System 

•  MALD  is  a  small,  low-cost,  expendable,  air-launched  vehicle 
that  replicates  how  fighter,  attack,  and  bomber  aircraft  appear 
to  enemy  radar  operators.  The  Air  Force  will  no  longer 
procure  any  MALDs,  as  the  MALD  procurement  line  was 
converted  to  MALD-J. 

•  The  Air  Force  designed  the  MALD-J  as  an  expendable, 
close-in  jammer  to  degrade  and  deny  an  early  warning  or 


acquisition  radar’s  ability  to  establish  a  track  on  strike  aircraft 
while  maintaining  the  ability  to  fulfill  the  MALD  decoy 
mission. 

•  The  F-16  C/D  and  B-52  are  the  lead  aircraft  to  employ  MALD 
and  MALD-J. 

Mission 

Combatant  Commanders  will  use  units  equipped  with: 

•  MALD  and  MALD-J  to  improve  battlespace  access  for 
airborne  strike  forces  by  deceiving,  distracting,  or  saturating 
enemy  radar  operators  and  integrated  Air  Defense  Systems. 

•  MALD  to  allow  an  airborne  strike  force  to  accomplish  its 
mission  by  forcing  enemy  radars  and  air  defense  systems  to 
treat  MALD  as  a  viable  target. 

•  MALD-J  to  allow  an  airborne  strike  force  to  accomplish  its 
mission  by  jamming  enemy  radars  and  air  defense  systems  to 
degrade  or  deny  detection  of  friendly  aircraft  or  munitions. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

MALD 

•  In  August  2011,  the  Air  Force  identified  a  fault  with  the 
missile’s  radio  frequency  connector  that  caused  it  to 
separate  from  the  missile  during  long-endurance  carriage 
flights.  After  improving  the  connector  system,  the  Air 
Force  tested  MALD  with  six  additional  shots  under  a 
Reliability  Assessment  Program  throughout  FY12. 

•  The  Air  Force  will  no  longer  procure  any  MALDs,  as  the 
Program  Office  converted  the  MALD  procurement  line  to 
MALD-J. 


MALD-J 

•  In  March  2012,  the  Air  Force  completed  the  MALD-J  EMD 
with  one  additional  test  mission  to  ensure  Operational 
Flight  Software  (OFS)  Build-7a  operated  successfully  and 
corrected  all  software  anomalies  found  with  the  EMD. 

•  DOT&E  approved  the  MALD-J  IOT&E  plan  in  May  2012. 

•  AFOTEC  launched  four  MALD-Js  during  IOT&E  in 
August  2012. 
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•  The  Air  Force  has  conducted  MALD-J  testing  to  date  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan  and  test  plan. 

•  The  Air  Force  is  currently  working  on  modeling  and 
simulation  utilizing  the  Digital  Integrated  Air  Defense 
System  to  evaluate  MALD’s  ability  to  degrade  an  Integrated 
Air  Defense  System.  Completion  and  final  verification  is 
scheduled  for  January  2013. 

Assessment 

•  Limited  accessibility  to  test  ranges,  unavailability  of  threat 

systems,  and  delays  in  processing  and  evaluating  data  have 

hampered  MALD  and  MALD-J  testing. 

MAID 

•  The  DOT&E  assessment  of  MALD  performance  in  the 
April  2011  MALD  IOT&E  Report  remains  unchanged. 
MALD  performance  is  operationally  effective  for  combat, 
but  not  operationally  suitable  due  to  poor  materiel  reliability 
in  the  intended  operational  environment. 

•  The  six  additional  shots  under  the  Reliability  Assessment 
Program  demonstrated  no  additional  critical  failures. 
However,  the  MALD  reliability  point  estimate  that 
combines  free-flight  and  aircraft  long-endurance  carriage 
was  78  percent,  which  falls  short  of  the  threshold 
requirement  of  93  percent.  This  reliability  shortfall  will 
increase  the  number  of  MALDs  necessary  to  accomplish  the 
mission. 

MALD-J 

•  DOT&E  conclusions  regarding  MALD-J  suitability, 
particularly  for  reliability,  depend  in  part  upon  data  from 


MALD  testing.  DOT&E  will  use  a  combination  of  MALD 
and  MALD-J  data  to  evaluate  whether  the  Air  Force  has 
resolved  reliability  problems.  After  completing  MALD-J 
EMD,  the  Air  Force  launched  14  MALD  and  MALD-Js 
during  FY12  without  additional  failures. 

•  Developing  a  full  mission-level  simulation  (i.e.,  multiple 
MALD-Js  versus  multiple  threat  radars)  is  a  technical 
challenge.  However,  the  oversight  of  stakeholders  and 
key  leadership  has  helped  the  Air  Force  to  continue 
development  of  the  simulation  capability  in  support  of  the 
AFOTEC  MALD-J  IOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
is  satisfactorily  addressing  four  of  the  five  FY 1 1 
recommendations.  However,  the  remaining  FY11 
recommendation  for  the  Air  Force  to  provide  sufficient 
resources  to  the  Nevada  Test  and  Training  Range  to  enable 
personnel  to  process  and  distribute  test  data  in  a  timely  manner 
requires  continued  emphasis. 

•  FY12  Recommendation. 

1 .  Future  strike  aviation  programs  should  consider  utilizing 
the  Air  Force  Digital  Integrated  Air  Defense  System 
modeling  and  simulation  capability  to  accurately  model 
the  operational  effect  of  MALD/MALD-J  and  other  future 
weapons  systems  in  robust  scenarios. 
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Mission  Planning  Systems  (MPS)  / 

Joint  Mission  Planning  Systems  -  Air  Force  (JMPS-AF) 


Executive  Summary 

•  The  Air  Force  paused  IOT&E  of  the  E-8  Mission  Planning 
Environment  (MPE),  the  representative  test  platform 

for  the  Joint  Mission  Planning  System  -  Air  Force 
(JMPS-AF)  Increment  IV  mission  planning  functionality,  in 
September  2011  to  allow  the  Program  Office  to  develop  and 
integrate  corrective  actions  to  deficiencies  identified  during 
operational  testing. 

•  In  January  2012,  the  DOT&E  Major  Automated  Information 
System  (MAIS)  report  documented  deficiencies  in  the 

E-8  MPE  version  1 .0  realized  during  the  suspended  F  Y 1 1 
IOT&E.  These  deficiencies  included  incomplete  printed 
flight  plans,  errors  in  flight  calculations  for  station  magnetic 
variation,  problems  with  exporting  mission  flight  plans, 
inadequate  training  for  intelligence  specialists,  unreliable  and 
time-consuming  system  setup  and  installation,  and  excessive 
time  needed  for  routine  maintenance. 

•  Following  additional  development  and  regression  testing,  the 
Air  Force  certified  E-8  MPE  version  1.3  ready  for  resumed 
operational  testing.  Air  Force  Operational  Test  and  Evaluation 
Center  (AFOTEC)  will  re-execute  the  entire  IOT&E  in  early 
FY13  in  accordance  with  the  DOT&E-approved  test  plan. 

System 

•  A  Mission  Planning  System  (MPS)  is  a  Standard  Desktop 
Configuration  (SDC)-based  common  solution  for  Air  Force 
aircraft  mission  planning  (the  current  SDC  is  Windows 
XP®  or  Vista®  PC -based).  It  is  a  package  of  common  and 
platform-unique  mission  planning  applications. 

•  An  MPE  is  a  set  of  developed  applications  built  from  a 
Framework,  common  components,  and  Unique  Planning 
Components  for  a  particular  aircraft.  The  Framework  is 
the  basis  of  the  MPE.  Software  developers  add  common 
components  (e.g.,  Weather,  Electronic  Warfare  Planner, 
etc.)  and  federated  applications  that  support  multiple  users 
to  the  framework.  Developers  then  add  a  Unique  Planning 
Component  for  the  specific  aircraft  type  (e.g.,  E-8)  to  complete 
the  MPE. 


•  The  MPE  can  operate  as  an  unclassified  system  or  a  classified 
system. 

•  Although  the  Framework  software  is  being  co-developed 
among  DoD  components,  MPS  is  not  a  joint  program.  Each 
Service  tests  and  fields  its  own  aircraft-specific  MPEs. 

•  The  representative  test  platform  for  JMPS-AF  Increment  IV 
mission  planning  functionality  is  the  E-8  MPE. 

Mission 

Aircrews  use  MPS  to  conduct  detailed  mission  planning  to 
support  the  full  spectrum  of  missions,  ranging  from  simple 
training  to  complex  combat  scenarios.  Aircrews  save  the 
required  aircraft,  navigation,  threat,  and  weapons  data  on  a  data 
transfer  device  that  they  load  into  their  aircraft  before  flight. 

Major  Contractors 

•  Northrop  Grumman  -  Carson  City,  California 

•  Boeing  -  St.  Louis,  Missouri 

•  TYBRIN  Corporation  -  Fort  Walton  Beach,  Florida 


Activity 

•  The  Air  Force  conducted  all  MPE  operational  testing  in 
accordance  with  a  DOT&E-approved  Test  and  Evaluation 
Master  Plan. 

•  AFOTEC  did  not  complete  IOT&E  of  the  E-8  MPE 
version  1.0  due  to  deficiencies  in:  navigation  functionality; 
magnetic  variation  computation;  mission  loading;  take-off 


and  landing  computations;  inadequate  intelligence  specialist 
training;  unreliable  system  set  up/installation;  and  excessive 
time  needed  for  routine  maintenance.  AFOTEC  returned  the 
system  to  development  for  corrective  action. 

•  The  46th  Test  Squadron  at  Eglin  AFB,  Florida,  conducted 
developmental  regression  testing  of  E-8  MPE  version  1.3  from 
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June  through  August  2012.  The  purpose  of  this  test  was  to 
verify  deficiency  fixes,  identify  changes  from  the  E-8  JMPS 
Version  1.0  MPE  Report,  and  determine  E-8  JMPS  version  1.3 
MPE  readiness  to  resume  IOT&E. 

•  Based  on  results  from  the  E-8  MPE  regression  testing, 
the  Air  Force  certified  E-8  MPE  version  1.3  ready  for 
IOT&E  in  September  2012.  AFOTEC  will  re-execute  the 
IOT&E  beginning  in  early  FY13  in  accordance  with  the 
DOT&E-approved  test  plan. 

Assessment 

•  In  February  2012,  DOT&E  published  a  MAIS  report 
concluding  that  E-8  MPE  version  1.0  was  operationally 
suitable  with  limitations  but  not  operationally  effective. 
Significant  findings  included: 

-  The  time  needed  for  E-8  MPE  software  installation  was 
lengthy,  due  in  large  part  to  anomalies  in  the  software 
functionality  and  installation  process. 

-  Threat  database  information  was  not  easily  accessible  or 
usable.  E-8  MPE  training  for  intelligence  specialists  was 
inadequate. 

-  While  mission  planners  could  use  E-8  MPE  version  1 .0  to 
generate  timely  mission  plans,  they  were  unable  to  transfer 
some  of  these  mission  plans  to  the  aircraft. 

-  The  MPE  generated  magnetic  variation  errors  for 
user-specified  waypoints.  Developers  identified  critical 
calculation  errors  of  the  magnetic  variation  errors  and 


incorporated  fixes  during  2012  developmental  testing. 
AFOTEC  will  verify  these  corrections  during  the  E-8  MPE 
version  1.3  IOT&E. 

-  Mission  planners  could  not  plan  missions  with  in-flight 
delays.  Developers  incorporated  fixes  during  2012 
developmental  testing  and  AFOTEC  will  verify  these 
corrections  during  the  E-8  MPE  version  1.3  IOT&E. 

-  The  Take-Off  and  Landing  Data  (TOLD)  module  in  E-8 
MPE  version  1 .0,  as  well  as  in  other  platform  MPEs  tested 
and  evaluated,  does  not  generate  accurate  data  and  is  not 
certified  for  flight  use.  The  Automated  TOLD  capability 
is  not  integrated  into  version  1.3,  and  the  Air  Force  has 
deferred  incorporation  of  this  objective  capability  to  a 
future  MPE  release. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  is 
addressing  previous  recommendations.  However,  the  Air 
Force  has  yet  to  incorporate  automated  TOLD  capabilities  into 
the  MPE. 

•  FY12  Recommendations.  The  Air  Force  should: 

1 .  Incorporate  automated  TOLD  capabilities  into  the  E-8 
MPE. 

2.  Re-execute  the  entire  E-8  MPE  IOT&E  in  accordance 
with  the  DOT&E-approved  test  plan  in  order  to  verify 
corrective  actions  and  determine  the  system’s  operational 
effectiveness,  suitability,  and  mission  capability. 
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MQ-9  Reaper  Armed  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  MQ-9  Unmanned  Aircraft  System  (UAS)  continues  to 
support  ongoing  global  combat  operations.  The  MQ-9  UAS 
has  met  urgent  combat  operational  needs  through  accelerated 
production  and  the  rapid  incorporation  of  emergent  sensor 
and  systems  technologies  outside  of  the  MQ-9  baseline 
program  of  record. 

•  The  Air  Force  accelerated  the  planned  FY13  Milestone  C 
Decision  for  the  Block  5  remotely-piloted  aircraft  (RPA) 
to  September  2012.  USD(AT&L)  approved  Block  5 
RPA  Milestone  C  and  Low-Rate  Initial  Production  in 
September  2012  and  the  MQ-9  UAS  Increment  One 
designation  as  an  Acquisition  Category  1C  program  in 
conjunction  with  the  Milestone  C  decision. 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  an  operational  assessment  (OA) 
of  the  Increment  One  UAS  to  inform  the  Block  5  RPA 
Milestone  C  decision.  A  limited  flight  demonstration 
comprising  5  functional  capabilities  sorties  and  18  flight 
test  hours  demonstrated  basic  integration  and  functionality 
of  the  Block  5  RPA  configuration  to  inform  the  Milestone  C 
decision. 

•  Ongoing  developmental  challenges  precluded  operational 
testing  and  subsequent  fielding  of  baseline  program  enhanced 
capabilities  to  operational  MQ-9  units  in  FY12  (Operational 
Flight  Programs  (OFP)  904.2,  904.4,  and  Joint  Direct  Attack 
Munition  (JDAM)). 

System 

•  The  UAS  includes  both  the  MQ-9  RPA  and  a  Ground  Control 
Station  (GCS). 

-  The  MQ-9  RPA  is  a  remotely-piloted,  armed  air  vehicle 
that  uses  optical,  infrared,  and  radar  sensors  to  locate, 
identify,  target,  and  attack  ground  targets.  The  RPA  is  a 
medium-sized  aircraft  that  has  an  operating  ceiling  up  to 
50,000  feet,  an  internal  sensor  payload  of  800  pounds,  an 
external  payload  of  3,000  pounds,  and  an  endurance  of 
approximately  14  hours. 

-  The  GCS  provides  aircraft  launch/recovery  and  mission 
control  of  sensors  and  weapons.  C-band  line-of-sight 
datalinks  are  used  for  RPA  launch  and  recovery 
operations,  and  Ku-band  satellite  links  are  used  for  RPA 
mission  control. 


•  The  MQ-9  RPA  carries  AGM-1 14,  Hellfire  II  anti-armor 
precision  laser-guided  missiles  and  GBU-12,  500  pound 
laser-guided  bombs. 

•  The  Air  Force  is  using  an  evolutionary  acquisition  approach  for 
meeting  Increment  One  Capability  Production  Document  (CPD) 
requirements,  with  Block  1  and  Block  5  RPAs  and  Block  15  and 
Block  30  GCSs. 

•  The  Air  Force  is  currently  fielding  the  Block  1  RPA. 

•  The  Air  Force  designed  the  Block  5  RPA  to  incorporate 
improved  main  landing  gear,  an  upgraded  electrical  system  with 
more  power,  an  additional  ARC-210  radio,  encrypted  datalinks, 
a  redesigned  avionics  bay  and  digital  electronic  engine  control 
system,  the  BRU-71  bomb  rack,  high-definition  video,  and 
upgraded  software  to  allow  the  two-person  aircrew  to  operate  all 
onboard  systems. 

Mission 

•  The  Combatant  Commander  uses  the  MQ-9  onboard  sensors 
and  weapons  to  conduct  armed  reconnaissance  and  pre-planned 
strikes.  Units  equipped  with  MQ-9s  can  find,  fix,  track,  target, 
engage,  and  assess  critical  emerging  targets  (both  moving  and 
stationary). 

•  MQ-9  units  can  also  conduct  aerial  intelligence  gathering, 
reconnaissance,  surveillance,  and  target  acquisition  for  other 
airborne  platforms. 

Major  Contractor 

General  Atomics  Aeronautical  Systems  Inc.  -  San  Diego,  California 


Activity 

•  The  Air  Force  conducted  MQ-9  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan. 

•  In  July  2012,  the  Air  Force  Requirements  Oversight  Council 
approved  changes  to  the  MQ-9  reliability  requirements.  The 


Air  Force  changed  the  Mean  Time  Between  Critical  Failure 
threshold  requirement  from  500  hours  to  19  hours  for  the 
MQ-9  RPA  and  from  500  hours  to  224  hours  for  the  GCS. 
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•  MQ-9  Block  1  RPA  software,  Block  5  RPA  hardware  and 
software,  and  Block  30  GCS  developmental  testing  were 
ongoing  throughout  FY12.  The  final  MQ-9  Increment  One 
UAS  configuration  will  include  the  Block  5  RPA,  Block  30 
GCS,  and  OFP  904.6. 

•  Planned  FY12  Air  Force  Air  Combat  Command  (ACC)  Force 
Development  Evaluation  (FDE)  testing  of  MQ-9  Block  1  OFPs 
904.2  and  904.4  did  not  occur.  Software  immaturity  coupled 
with  the  implementation  of  new  Air  Force  airworthiness 
requirements  resulted  in  developmental  delays. 

•  The  2009  GBU-38,  500-pound  JDAM  FDE  to  support  limited 
MQ-9  fielding  remains  in  a  pause  status  pending  resolution 
of  MQ-9  OFP  fuzing  and  weapons  envelope  discrepancies 
identified  in  2010. 

•  The  new  Block  5  RPA  completed  five  early  developmental 
demonstration  flights  in  FY12.  These  functional  capabilities 
sorties  demonstrated  basic  integration  and  functionality  of  the 
Block  5  RPA  configuration. 

•  AFOTEC  began  an  OA  of  MQ-9  in  May  2012  in  support  of 
the  September  2012  Block  5  Milestone  C  decision.  AFOTEC 
conducted  the  testing  in  accordance  with  a  DOT&E-approved 
OA  plan. 

•  USD(AT&L)  approved  MQ-9  Block  5  RPA  Milestone  C  and 
low-rate  initial  production  in  September  2012. 

Assessment 

•  The  Air  Force  intends  to  fulfill  the  MQ-9  Increment  One  CPD 
requirements  with  a  final  UAS  configuration  consisting  of 
the  Block  5  RPA,  Block  30  GCS,  and  OFP  904.6.  AFOTEC 
will  conduct  formal  operational  testing  of  the  final  MQ-9 
Increment  One  UAS  in  2014.  This  operational  testing  will 
assess  Increment  One  UAS  effectiveness,  suitability,  mission 
capabilities,  and  satisfaction  of  CPD  key  performance 
parameters. 

•  The  MQ-9  program  continues  to  face  systemic  challenges  in 
prioritizing  and  maturing  software  OFPs  to  meet  development 
and  fielding  timelines  for  the  Increment  One  program  of 
record.  During  FY12,  such  delays  precluded  the  completion 
of  developmental  testing  of  OFPs  904.2  and  904.4  to  support 
planned  FY12  operational  testing.  Development,  operational 
testing,  and  fielding  of  Increment  One  program  of  record 
capabilities  will  likely  experience  continued  delays  unless  the 
program  is  able  to  better  prioritize  and  control  maturation  of 
these  capabilities. 


•  The  program  made  incremental  progress  towards  resolving 
deficiencies  discovered  during  the  2009  MQ-9  JDAM  FDE 
testing;  however,  the  OFP  has  yet  to  demonstrate  readiness  for 
resumed  JDAM  operational  testing. 

•  Findings  from  the  AFOTEC  OA  of  the  Increment  1  Block  5 
RPA  indicated  progress  in  conducting  limited  flight  testing  of 
the  integrated  hardware  suite  and  early  revisions  of  OFP  904.6 
software  in  a  prototype  Block  5  RPA  controlled  with  a  Block 
15  GCS.  Across  5  sorties  and  18  flight  test  hours  these 
missions  demonstrated: 

-  Functional  operation  of  aircraft  handling,  flight  control 
systems,  and  payload  systems 

-  Successful  operation  of  the  heavyweight  landing  gear 
system 

-  All  power  modes  of  the  high  capacity  power  system 

-  Dual  ARC-210  radio  and  wingtip  antenna  functionality  in 
both  clear  and  secure  modes 

-  Encrypted  Ku-band  satellite  datalink  control 

-  Multi-spectral  Targeting  System  B  functionality  in  standard 
definition  video  format 

-  Synthetic  aperture  radar  legacy  modes,  including  ground 
moving  target  indicator  mode  operation 

•  As  was  the  case  in  FY11,  Information  Assurance  (IA) 
vulnerabilities  and  deficiencies  are  not  well  characterized 
because  the  Air  Force  has  only  completed  limited  IA  testing 
on  the  MQ-9  system.  As  of  the  end  of  FY12,  the  system  is 
operating  under  an  Interim  Authority  to  Test,  pending  full 
system  IA  testing. 

Recommendations 

•  Status  of  Previous  Recommendations.  In  FY12,  the  Air  Force 
satisfied  several  of  the  DOT&E  FY11  recommendations. 
However,  two  FY11  recommendations  remain  outstanding: 

1 .  The  Air  Force  should  complete  the  JDAM  FDE. 

2.  The  Air  Force  should  conduct  IA  testing  to  include 
adversary  system  penetration  testing. 

•  FY12  Recommendation 

1.  The  Air  Force  should  complete  the  development  of  the 
Increment  One  UAS  hardware  and  software  to  support 
FOT&E  of  the  Increment  One  system  to  assess  operational 
effectiveness,  suitability,  and  mission  capability  of  the  final 
Increment  One  UAS  configuration. 
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RQ-4B  Global  Hawk  High-Altitude  Long-Endurance 
Unmanned  Aerial  System  (UAS) 


Executive  Summary 

Block  20 

•  The  Air  Force  successfully  completed  conversion  of  two 
additional  RQ-4B  Block  20  air  vehicles  to  the  EQ-4B 
configuration  equipped  with  the  Battlefield  Airborne 
Communications  Node  (BACN)  payload.  This  increased 
the  total  number  of  BACN-equipped  air  vehicles  supporting 
U.S.  Central  Command  (USCENTCOM)  operations  to 
three.  Developmental  and  operational  testing  of  RQ-4B 
and  EQ-4B  Block  20  Global  Hawk  variants  are  complete. 

Block  30 

•  The  Air  Force  continued  developmental  testing  of  RQ-4B 
Block  30  capability  upgrades  and  deficiency  corrections 
at  a  reduced  pace  due  to  the  FY13  budget  proposal  to 
terminate  the  program.  The  Air  Force  continued  to  improve 
RQ-4B  Block  availability  by  correcting  air  vehicle  and 
sensor  reliability  problems  and  increasing  the  availability  of 
spare  parts. 

•  The  Air  Force  has  not  conducted  an  RQ-4B  Block  30 
operational  test  to  field  deficiency  corrections  or  previously 
planned  new  system  capabilities  to  operational  units 
since  IOT&E  in  December  2010.  Future  plans  for  system 
development  and  testing  depend  upon  final  FY13  budget 
decisions. 

Block  40 

•  The  Air  Force  completed  the  initial  phase  of  RQ-4B 
Block  40/Multi-Platform  Radar  Technology  Insertion 
Program  (MP-RTIP)  radar  system-level  performance 
verification  testing  in  July  2012,  which  demonstrated  basic 
integration  of  the  MP-RTIP  radar  with  the  Global  Hawk 
air  vehicle.  This  phase  also  focused  on  improving  radar 
software  stability,  image  quality,  and  moving  target  false 
alarm  rates. 

•  In  response  to  a  USCENTCOM  early  operational  need 
request,  the  Air  Force  is  planning  a  limited  operational 
utility  evaluation  in  March  2013  to  support  early  fielding 
of  two  developmental  RQ-4B  Block  40  systems  to  support 
USCENTCOM  operations. 

System 

•  The  RQ-4  Global  Hawk  is  a  remotely-piloted,  high-altitude, 
long-endurance  airborne  intelligence,  surveillance, 

and  reconnaissance  system  that  includes  the  Global 
Hawk  unmanned  air  vehicle,  various  intelligence  and 
communications  relay  mission  payloads,  and  supporting 
command  and  control  ground  stations. 

•  The  RQ-4B  Global  Hawk  Block  20  system  is  equipped 
with  the  Enhanced  Integrated  Sensor  Suite  (EISS)  imagery 
intelligence  payload.  The  EQ-4B  Block  20  variant  is  equipped 


with  the  BACN  theater  communications  relay  payload.  The 
EISS  sensor  includes  infrared,  optical,  and  synthetic  aperture 
radar  sensors  for  collecting  still  imagery  intelligence  on 
ground  targets.  The  BACN  payload  provides  communications 
connectivity  between  geographically  separated  operational 
units. 

•  The  RQ-4B  Global  Hawk  Block  30  system  is  equipped  with  a 
multi-intelligence  payload  that  includes  both  the  EISS  imagery 
intelligence  payload  and  the  Airborne  Signals  Intelligence 
Payload  (ASIP)  electronic  signal  collection  sensor. 

•  The  RQ-4B  Global  Hawk  Block  40  system  is  equipped  with 
the  MP-RTIP  synthetic  aperture  radar  payload  designed  to 
simultaneously  collect  imagery  intelligence  on  stationary 
ground  targets  and  track  ground  moving  targets. 

Mission 

Commanders  use  RQ-4  Global  Hawk  reconnaissance  units  to 
provide  high-altitude,  long-endurance  intelligence  collection 
capabilities  or  theater  communications  relay  capabilities  to 
supported  commanders. 

•  Operators  collect  imagery  and  signals  data  in  order  to  support 
ground  units  and  to  identify  intelligence  essential  elements 
of  information  for  theater  commanders.  Units  equipped 
with  RQ-4B  Global  Hawk  use  line-of-sight  and  beyond 
line-of-sight  satellite  datalinks  to  control  the  Global  Hawk 
system  and  transmit  collected  intelligence  data. 

•  Distributed  intelligence  processing,  exploitation,  and 
dissemination  systems  receive  intelligence  data  directly  from 
the  air  vehicle  or  from  the  Global  Hawk  ground  station  via 
intelligence  data  transmission  systems. 
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•  Ground-based  intelligence  analysts  exploit  collected  imagery 
and  signals  information  to  provide  intelligence  products  in 
support  of  theater  operations. 

•  Global  Hawk  can  also  provide  imagery  intelligence  directly  to 
forward-based  personnel  through  direct  line-of-sight  datalink 
systems. 


Major  Contractor 

Northrop  Grumman  Aerospace  Systems,  Strike  and  Surveillance 
Systems  Division  -  San  Diego,  California 


Activity 

Block  20 

•  In  response  to  a  USCENTCOM  request,  the  Air 
Force  successfully  completed  conversion  of  two 
additional  RQ-4B  Block  20  air  vehicles  to  the  EQ-4B 
configuration  equipped  with  the  BACN  payload  to 
support  USCENTCOM  operations.  The  Air  Force 
completed  testing,  and  deployed  these  modified  systems  to 
USCENTCOM  in  FY12. 

•  Three  of  the  four  operational  EQ-4B  Block  20  air  vehicles 
are  now  supporting  USCENTCOM  operations  with  the 
BACN  payload.  The  fourth  aircraft  is  a  RQ-4B  test  asset. 
Developmental  and  operational  testing  of  this  Global  Hawk 
variant  is  complete. 

Block  30 

•  Due  to  the  FY 13  budget  proposal  to  terminate  RQ-4B 
Block  30  production  and  retire  all  existing  aircraft, 
USD(AT&L)  deferred  the  planned  RQ-4B  Block  30 
Full-Rate  Production  Decision  from  August  2011  until  at 
least  November  2012. 

•  The  Air  Force  continued  developmental  testing  of  software 
improvements  to  correct  operational  deficiencies  identified 
during  the  FY11  IOT&E.  However,  the  Air  Force  did  not 
complete  planned  RQ-4B  Block  30  Force  Development 
Evaluation  follow-on  operational  testing  in  FY11  or  FY12 
to  evaluate  and  field  these  improvements  due  to  uncertainty 
resulting  from  the  FY13  budget  proposal  to  terminate  the 
program.  The  Air  Force  continued  to  implement  corrective 
actions  to  improve  system  reliability,  availability,  and 
maintainability  for  the  RQ-4B  Block  30  systems  currently 
employed  in  USCENTCOM,  U.S.  Pacific  Command,  and 
U.S.  European  Command  operational  theaters. 

•  The  Air  Force  continued  planning  for  an  RQ-4B  Block  30 
Force  Development  Evaluation  operational  test  in  FY13 
to  evaluate  specific  post-IOT&E  air  vehicle  and  sensor 
software  upgrades  and  a  new  satellite  communications 
link.  Planning  for  a  more  comprehensive  FY14  RQ-4B 
Block  30/ASIP  FOT&E  event  to  evaluate  correction  of 
all  major  system  deficiencies  identified  during  IOT&E 
remains  on  hold  pending  a  final  decision  on  the  future  of 
the  RQ-4B  Block  30  program. 

Block  40 

•  Due  to  FY13  budget  uncertainties,  USD(AT&L)  deferred 
the  RQ-4B  Global  Hawk  Block  40  Milestone  C  Decision 
from  August  2011  until  FY13. 


•  In  preparation  for  a  Milestone  C  decision,  the  Air  Force 
is  developing  a  revised  RQ-4B  Block  40  Capability 
Production  Document  for  Joint  Staff  approval,  which 
removes  Battle  Management  Command  and  Control 
mission  capabilities  as  a  threshold  requirement  and  lowers 
overall  system  reliability  and  availability  requirements. 

The  Air  Force  also  initiated  development  of  an  RQ-4B 
Block  40  Milestone  C  Test  and  Evaluation  Master  Plan  to 
guide  system  developmental  and  operational  test  activities 
leading  to  a  combined  RQ-4B  Block  40/MP-RTIP  IOT&E 
in  FY14  and  FOT&E  of  future  capabilities  and  system 
improvements. 

•  The  Air  Force  continued  execution  of  the  current  RQ-4B 
Block  40/MP-RTIP  development  and  test  program. 

The  Air  Force  completed  the  first  phase  of  RQ-4B 
Block  40/MP-RTIP  radar  system-level  performance 
verification  testing  in  July  2012  to  verify  basic  integration 
of  the  MP-RTIP  radar  with  the  air  vehic  le  platform. 
Additional  radar  control,  performance,  and  interoperability 
testing  is  in  progress  and  will  continue  until  IOT&E 
in  FY14. 

•  In  response  to  a  USCENTCOM  early  operational  need 
request,  the  Air  Force  is  planning  a  limited  operational 
utility  evaluation  in  March  2013  to  support  early  fielding 
of  two  developmental  RQ-4B  Block  40  systems  to  support 
USCENTCOM  operations. 

•  Accelerated  delivery  of  RQ-4B  Block  40  air  vehicles  and 
MP-RTIP  sensor  systems  continued  in  FY12.  The  Air  Force 
will  accept  delivery  of  at  least  9  of  the  1 1  procured  RQ-4B 
Block  40  systems  at  Grand  Forks  AFB,  North  Dakota,  prior 
to  IOT&E  and  initial  operational  capability  in  FY14. 

Assessment 

Block  20 

•  Correction  of  previously  identified  EQ-4B  air  vehicle 
reliability  problems  and  procurement  of  additional  spare 
parts  increased  the  system  reliability  and  availability. 

Block  30 

•  Due  to  FY13  budget  uncertainties  and  lack  of  planned 
follow-on  operational  testing,  the  Air  Force  has  not  fielded 
software  deficiency  corrections  or  previously  planned 
additional  capabilities  to  operational  units.  Future  plans  for 
system  development  and  testing  depend  upon  final  FY13 
budget  decisions. 
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•  The  Air  Force  continued  to  improve  RQ-4B  Block  30 
reliability  and  availability  by  correcting  air  vehicle  and 
sensor  reliability  problems  and  increasing  the  availability 
of  spare  parts.  Maintenance  training  and  technical  order 
improvements  also  improved  system  maintainability.  As 
a  result,  RQ-4B  Block  30  mission  capable  rates  increased 
from  approximately  52  percent  during  the  FY11  IOT&E  to 
over  80  percent  in  FY12.  At  these  rates,  RQ-4B  Block  30 
effective-time-on-station  performance  at  near  continuous 
operational  tempos  should  be  sufficient  to  meet  the 
minimum  55  percent  effective-time-on-station  operational 
requirement  for  single  vehicle  operations,  if  sufficient 
spare  parts  remain  available. 

Block  40 

•  The  RQ-4B  Block  40  and  MP-RT1P  development  program 
made  significant  progress  in  FY12.  The  initial  phase  of 
radar  system-level  performance  verification  test  in  July 
2012  demonstrated  basic  integration  of  the  MP-RTIP  radar 
with  the  air  vehicle.  This  phase  also  focused  on  improving 
radar  software  stability,  image  quality,  and  moving  target 
false  alarm  rates 

•  The  RQ-4B  Block  40/MP-RTIP  test  schedule  to  support 
early  fielding  of  two  systems  to  USCENTCOM  in 
May  2013  and  the  FY14  IOT&E  is  aggressive  with  little 
room  for  recovery  if  ongoing  developmental  tests  discover 
any  significant  system  performance  problems.  Ongoing 
sensor  data  transfer  and  interoperability  challenges  and 
potential  winter  weather  delays  at  the  Grand  Forks, 

North  Dakota,  test  site  further  increase  schedule  risks. 


Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  made 
progress  in  addressing  previous  DOT&E  recommendations  to 
implement  RQ-4B  Block  30  system  reliability,  interoperability, 
and  Information  Assurance  improvements.  The  Air  Force 
delayed  full  implementation  of  the  ASIP  sensor  performance 
improvement  plans  and  other  post-IOT&E  RQ-4B  Block  30 
improvements  due  to  the  FY13  budget  proposal  to  terminate 
the  program.  The  Air  Force  successfully  addressed  DOT&E 
recommendations  to  revise  RQ-4B  Block  40  operational 
capability  requirements  and  define  related  end-to-end 
operational  architectures  and  interoperability  requirements. 

•  FY 12  Recommendations.  The  Air  Force  should: 

1.  Develop  an  RQ-4B  Block  30  test  strategy  to  complete 
post-IOT&E  corrective  actions  and  conduct  a 
comprehensive  FOT&E  if  the  decision  is  made  to  continue 
the  program.  If  terminated,  the  Air  Force  should  conduct 
FOT&E  of  the  ASIP  sensor  on  the  U-2  aircraft  to  verify 
correction  of  serious  performance  deficiencies  identified 
during  IOT&E. 

2.  Conduct  an  operational  test  to  support  early  fielding  of 
RQ-4B  Block  40  developmental  systems  to  meet  the  FY13 
USCENTCOM  operational  need  request. 

3.  Complete  an  RQ-4B  Block  40  Test  and  Evaluation  Master 
Plan  to  guide  system  test  activities  leading  to  a  combined 
RQ-4B  Block  40/MP-RTIP  IOT&E  in  FY14  and  the 
required  follow-on  developmental  and  operational  testing 
of  planned  future  Global  Hawk  capabilities  and  system 
improvements. 
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Space-Based  Infrared  System  (SBIRS)  Effectivity  5 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  an  Operational  Utility  Evaluation 
(OUE)  of  the  Space-Based  Infrared  System  (SBIRS)  from 
September  27  to  October  11,  2012.  The  capability  tested  was 
Effectivity  5,  which  includes  the  SBIRS  ground  architecture, 
the  first  SBIRS  satellite  in  geosynchronous  orbit  (GEO-1), 
two  hosted  infrared  payloads  in  Highly  Elliptical  Orbit  (HEO), 
and  legacy  Defense  Support  Program  (DSP)  assets.  DOT&E 
published  a  classified  test  report  in  December  2012. 

•  The  OUE  will  inform  Air  Force  operational  acceptance  of 
Effectivity  5.  U.S.  Strategic  Command  will  use  operational 
test  results  for  its  Integrated  Tactical  Warning/ Attack 
Assessment  system  certification.  The  National 
Geospatial-Intelligence  Agency  (NGA)  will  utilize  operational 
test  results  for  its  certification  of  GEO-1  data  for  technical 
intelligence. 

•  SBIRS  Effectivity  5  is  operationally  effective  and  will  be 
operationally  suitable  upon  resolution  of  an  open  deficiency 
identified  in  the  classified  DOT&E  report. 

System 

The  SBIRS  program  provides  infrared  sensing  from  space  to 

support  Department  of  Defense  and  other  user  organizations. 

The  SBIRS  program  is  being  developed  in  two  system 

increments  to  replace  the  DSP  satellites: 

•  Increment  1  uses  the  SBIRS  Control  Segment  and  User 
Segment,  operating  with  DSP  satellites,  to  provide  current 
military  capability.  Initial  Operational  Capability  for 
Increment  1  was  attained  in  December  2001,  consolidating  the 
operations  of  the  DSP  and  Attack  and  Launch  Early  Reporting 
to  Theater  missions. 

•  Increment  2  inc  ludes  a  space  segment  consisting  of  two  hosted 
payloads  in  HEO  and  four  satellites  in  geosynchronous  orbit. 
Increment  2  also  provides  new  software  and  hardware  to 
process  data  from  both  the  DSP  and  the  SBIRS  space  segment. 

•  Increment  2  capabilities  are  being  delivered  in  phases,  both 
with  ground  system  software  and  on-orbit  assets,  and  therefore 
require  several  dedicated  test  and  evaluation  activities.  The 


current  phase  tested,  Effectivity  5,  adds  capabilities  into  the 
operational  SBIRS  system  to  support  GEO-1,  which  was 
launched  on  May  7,  2011.  New  capabilities  include  GEO-1 
command  and  control  functions  and  use  of  GEO-1  scanner 
sensor  data  in  mission  processing  and  reporting  functions. 

Mission 

Combatant  Commanders,  deployed  U.S.  military  forces,  and 
allies  intend  to  use  SBIRS  to  conduct  missions  that  require 
improved  space  sensors  and  operational  launch  detection 
capabilities.  Commanders  will  use  SBIRS  to  enhance  support 
to  joint  combat  forces  in  four  key  areas: 

•  Timely  and  responsive  space-based  missile  warning  and 
detection 

•  Launch  detection  and  characterization  for  missile  defense 
operations 

•  Technical  intelligence 

•  Battlespace  awareness 

Major  Contractor 

Lockheed  Martin  Space  Systems  -  Sunnyvale,  California 


Activity 

•  AFOTEC  conducted  a  dedicated  OUE  from  September  27 
through  October  11,  2012.  AFOTEC  conducted  the  testing  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan  (TEMP)  and  test  plan.  The  OUE  was  executed  in 
conjunction  with  Air  Force  Space  Command’s  operational  trial 
period.  An  initial  period  occurred  from  September  27  through 
November  27,  2012.  An  additional  trial  period  is  projected  to 


occur  following  resolution  of  a  deficiency  affecting  suitability, 
which  is  discussed  in  the  classified  report. 

-  The  capability  tested  was  Effectivity  5,  which  includes  the 
SBIRS  ground  architecture,  GEO-1,  two  hosted  infrared 
payloads  in  HEO,  and  legacy  DSP  assets. 

-  DOT&E  published  a  final  test  report  in  December  2012. 
The  report  will  inform  Air  Force  operational  acceptance 
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of  Effectivity  5.  U.S.  Strategic  Command  will  also 
use  operational  test  results  for  its  Integrated  Tactical 
Warning  / Attack  Assessment  system  certification.  NGA 
will  utilize  operational  test  results  for  its  certification  of 
GEO-1  data  for  technical  intelligence. 

•  The  Air  Force  successfully  launched  SB1RS  GEO-1  on 
May  7,  2011,  and  conducted  planned  checkout  and  sensor 
tuning  activities  through  May  2012  for  the  strategic  and  theater 
missile  warning  missions.  Additional  checkout  activities  are 
planned  to  continue  throughout  FY13,  including  testing  and 
calibration  of  the  staring  sensor. 

•  DOT&E  approved  an  updated  Enterprise  TEMP  on 
March  7,  2012.  Another  update  to  the  TEMP  is  in 
coordination  for  future  testing  of  Increment  2.  Finalizing  this 
document  is  contingent  upon  a  ground  architecture  definition, 
a  concept  of  operations,  and  operational  requirements  for  each 
key  SB1RS  Increment  2  delivery. 

•  AFOTEC  and  the  SB1RS  Program  Office  led  an  Integrated 
Test  and  Evaluation  period  between  June  and  August  2012. 

The  Air  Force  made  modeling  and  simulation  requirements  a 
priority,  working  with  the  Program  Office  and  contractor  to 
ensure  required  capabilities  were  sustained  and  available  for 
Integrated  Test. 

Assessment 

•  SBIRS  Effectivity  5  is  operationally  effective.  Integration  of 
GEO-1  into  the  operational  constellation  improved  accuracy 
of  both  strategic  and  theater  missile  warning  mission  data 
and  did  not  degrade  overall  mission  performance.  SBIRS 
also  demonstrated  improved  performance  against  the  missile 
defense  mission.  SBIRS  support  to  the  technical  intelligence 
and  battlespace  awareness  missions  was  functional  and 
effective.  There  were  no  major  problems  observed  during  the 
integrated  and  operational  test  periods.  The  SBIRS  enterprise 


system  accomplished  its  strategic  and  theater  missile  warning 
missions,  successfully  detecting  and  reporting  all  missile 
events  during  both  real-world  and  simulation  scenarios  during 
these  test  periods. 

•  The  SBIRS  GEO- 1  scanning  sensor  payload  is  meeting 
accuracy  and  sensitivity  requirements,  based  on  developmental 
and  integrated  test  activities.  It  is  at  least  as  capable  as  legacy 
DSP  sensors,  while  providing  detection  over  a  given  location 
twice  as  frequently.  This  increased  revisit  rate  is  operationally 
significant  as  it  enables  the  ability  to  determine  target  missile 
type  with  higher  confidence  by  providing  more  data  points  for 
analysis  during  the  target  missile’s  powered  flight. 

•  SBIRS  Effectivity  5  will  be  operationally  suitable  upon 
resolution  of  an  open  deficiency  identified  in  the  classified 
DOT&E  OUE  report.  The  Air  Force  is  addressing  problems 
identified  during  the  OUE  with  the  overall  system,  technical 
intelligence  missions,  and  specific  Information  Assurance 
postures. 

•  The  classified  test  report  includes  more  information 
on  additional  observations,  detailed  findings,  and 
recommendations 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
satisfactorily  addressed  previous  recommendations  from 
FY05,  FY07,  FY08,  and  FY09. 

•  FY12  Recommendations.  The  Air  Force  should 

1 .  Specify  the  ground  architecture,  concept  of  operations,  and 
operational  requirements  for  each  key  SBIRS  Increment  2 
delivery  due  to  recent  SBIRS  restructuring  to  support  a 
timely  update  of  the  TEMP. 

2.  Address  the  eight  recommendations  identified  in  the 
December  2012  classified  DOT&E  OUE  report. 
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Executive  Summary 

•  In  October  2012,  the  Missile  Defense  Agency  (MDA) 
conducted  the  most  complex  ballistic  missile  defense 
flight  test,  combined  developmental/operational  Flight  Test 
Integrated-0 1  (FTI-01),  ever  attempted  in  the  history  of  the 
DoD.  FTI-01  was  the  first  flight  test  of  a  regional  Ballistic 
Missile  Defense  System  (BMDS)  that  included  soldiers, 
sailors,  and  airmen  from  multiple  Combatant  Commands 
operating  multiple  sensors  and  missile  defense  systems  to 
engage  three  ballistic  and  two  cruise  missile  targets  near 
simultaneously.  Four  of  the  five  targets  were  successfully 
intercepted.  Analysis  is  ongoing,  but  DOT&E  will  include  the 
results  in  the  2012  BMDS  report  to  Congress  to  be  published 
in  February  2013. 

•  Except  for  FTI-01,  testing  of  the  BMDS  during  the  past  fiscal 
year  was  limited  to  three  ground  tests,  two  Aegis  BMD  flight 
tests,  one  Terminal  High-Altitude  Area  Defense  (THAAD) 
flight  test,  and  exercise  Fast  Eagle  Increment  1  testing.  The 
Ground-based  Midcourse  Defense  (GMD)  program  did  not 
accomplish  any  element  flight  testing. 

•  BMDS  regional  defensive  capability  was  enhanced  by 
completion  of  FTI-01,  deployment  of  Phase  1  of  the  European 
Phased  Adaptive  Approach  (EPAA),  and  the  lessons  learned 
during  exercise  Fast  Eagle  Increment  1. 


•  The  MDA,  in  collaboration  with  the  Operational  Test 
community,  continued  to  refine  the  content  and  quality 
of  the  BMDS  Integrated  Master  Test  Plan  (IMTP)  with 
emphasis  on  the  verification,  validation,  and  accreditation 
of  the  models  and  simulations  supporting  performance 
assessments. 

System 

•  The  current  BMDS  architecture  integrates  ballistic  missile 
defense  capabilities  against  all  ranges  of  threats. 

•  BMDS  is  a  distributed  system  currently  comprised  of  five 
elements  (four  shooter  elements  and  one  command  and 
control  element)  and  five  sensor  systems  (four  radar  systems 
and  one  space-based  system). 

Elements 

•  Aegis  BMD  (shooter) 

•  Command,  Control,  Battle  Management,  and 
Communications  (C2BMC)  (command  and  control) 

•  GMD  (shooter) 

•  Patriot  (shooter) 

•  THAAD  (shooter) 
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Sensors 

•  Aegis  BMD  AN/SPY- 1  Radar 

•  Cobra  Dane  Radar 

•  Upgraded  Early  Warning  Radars 

•  AN/TPY-2  (Forward-Based  Mode  [FBM])  Radar 

•  Space  Based  Infrared  System/Defense  Support  Program 
(SB1RS/DSP) 

•  These  two  sensor  systems  will  support  the  BMDS  when 
available: 

-  Sea-Based  X-Band  Radar  (primarily  a  test  asset  that  can 
be  operationally  deployed  as  needed) 

-  Precision  Tracldng  Space  System  (a  projected  addition  to 
the  BMDS  sensor  systems) 

Mission 

•  The  U.S.  Strategic  Command  (USSTRATCOM)  synchronizes 
operational-level  global  missile  defense  planning,  operations 
support,  and  the  development  of  missile  defense  effects  for 
the  DoD.  When  directed,  it  provides  alternate  missile  defense 
execution. 

•  U.S.  Northern  Command  (USNORTHCOM),  U.S. 

Pacific  Command  (USPACOM),  U.S.  Central  Command 
(USCENTCOM),  and  U.S.  European  Command  (USEUCOM) 
will  employ  the  assets  of  the  BMDS  to  defend  U.S.  territory. 


deployed  forces,  friends,  and  allies  against  ballistic  missile 
threats  of  all  ranges  and  in  all  phases  of  flight.  Initial 
capability  permits  defending  U.S.  territory  against  simple 
ballistic  missile  threats  and  defending  deployed  forces,  friends, 
and  allies  from  theater-level  ballistic  missile  threats. 

•  USSTRATCOM,  USNORTHCOM,  USEUCOM, 
USCENTCOM,  and  USPACOM  will  use  the  C2BMC  element 
of  the  BMDS  to  maintain  situational  awareness.  USEUCOM, 
USCENTCOM,  and  USPACOM  will  also  use  the  C2BMC  to 
provide  sensor  management  of  theater  AN/TPY-2  radars  across 
the  full  mission  engagement  space. 

•  The  Army  employs  Patriot  to  provide  theater  defense  for 
deployed  forces  against  short-  and  medium-range  threats. 

Major  Contractors 

•  The  Boeing  Company,  Integrated  Defense  Systems,  Missile 
Defense  Systems  -  Huntsville,  Alabama 

•  Lockheed  Martin: 

-  Missile  and  Fire  Control  -  Dallas,  Texas 

-  Mission  Systems  and  Sensors  -  Moorestown,  New  Jersey 

-  Information  Systems  and  Global  Solutions  -  Gaithersburg, 
Maryland 

•  Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  The  MDA  conducted  FTI-01  in  October  2012,  which  included 
near-simultaneous  Aegis  BMD  and  Patriot  engagements  of 
short-range  ballistic  missiles  while  defending  against  cruise 
missile  attacks,  and  a  THAAD  first  time  engagement  of  a 
medium-range  ballistic  missile.  SBIRS/DSP  provided  early 
warning  and  an  AN/TPY-2  (FBM)  radar  provided  acquisition 
cues  via  C2BMC.  Soldiers  performed  command  and  control 
functions  from  the  Air  and  Space  Operations  Center  at  Hickam 
AFB,  Hawaii. 

•  In  FY12,  the  MDA  completed  the  Ground  Test-04d  (GT-04d) 
campaign,  which  consisted  of  Ground  Test  Integrated-04d 
Part  2  in  October  2011  and  Ground  Test  Distributed-04d 
(GTD-04d)  Parts  2  and  3  in  December  2011.  These  ground 
tests  provided  support  to  the  MDA’s  European  Phased 
Adaptive  Approach  (EPAA)-Phase  1  declaration  against 
medium-range  ballistic  missiles  and  NATO’s  May  2012 
declaration  of  an  interim  southern  European  missile  defense 
capability. 

•  The  MDA  completed  GT1-04  (Israel)  in  November  2011, 
which  tested  the  interoperability  of  Israel’s  joint  BMDS.  It 
integrated  the  following  U.S.  and  Israeli  systems:  C2BMC, 
Aegis  BMD,  AN/TPY-2  (FBM),  SBIRS/DSP,  THAAD,  and  a 
representation  of  the  Israeli  Arrow  Weapons  System. 

•  The  MDA  completed  Ground  Test  Focused-04e  (GTX-04e)  in 
April  2012.  This  developmental  ground  test  provided  data  to 
assess  new  functionality  for  Aegis  BMD,  GMD,  and  THAAD. 

•  The  MDA  completed  the  Fast  Eagle  Increment  1 
hardware-in-the-loop  testing  in  June  2012,  a  distributed  test 


in  August  2012,  and  follow-on  hardware-in-the-loop  testing 
in  September  2012.  These  tests  were  designed  to  evaluate 
the  capability  of  the  AN/TPY-2  (FBM)  and  USCENTCOM 
C2BMC  to  augment  the  existing  USCENTCOM  BMDS 
capability. 

•  The  MDA,  in  collaboration  with  the  Operational  Test 
community,  made  one  update  to  the  FY12  version  of  the 
BMDS  IMTP  that  uses  a  critical  factors  analysis  (referred  to 
as  Critical  Engagement  Conditions)  and  other  important  data 
needs  (referred  to  as  Empirical  Measurement  Events)  to  drive 
test  design,  planning,  and  execution.  In  July  2012,  the  MDA 
began  a  process  of  re-baselining  the  IMTP  to  better  align  it 
with  BMDS  modeling  and  simulation  verification,  validation, 
and  accreditation  data  needs.  The  MDA  conducted  testing 
during  FY12  in  accordance  with  the  DOT&E-approved  IMTP. 

•  In  F Y 1 1 ,  the  MDA  established  a  Lethality  Focus  Group 
(LFG)  in  which  the  MDA,  DOT&E,  and  the  Operational  Test 
community  collaborate  to  identify  lethality  data  gaps  for  all 
BMDS  weapon  elements.  In  FY12,  the  LFG  generated  a 
prioritized  list  of  threat  models  needed  to  support  lethality 
evaluations  for  planned  flight  and  ground  testing  for  Aegis 
BMD,  GMD,  Patriot,  and  THAAD,  and  has  started  developing 
a  plan  of  action  to  address  these  data  voids. 

•  During  FY12,  the  MDA  conducted  numerous  wargames 
and  exercises  that  enhanced  Combatant  Command  BMD 
readiness  and  increased  confidence  in  the  deployed  elements 
of  the  BMDS.  These  events  included:  Air  and  Missile 
Defense  Exercise  (AMDEX),  Joint  Air  and  Missile  Defense 
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Exercise,  Austere  Challenge,  Terminal  Fury/Global  Lightning, 
EUCOM  AMDEX,  Keen  Edge,  Ulchi  Freedom  Guardian, 
Vigilant  Shield/Global  Thunder,  BMDS  Wargame,  Planning 
Exercise,  Joint  Tactical  Air  Picture,  National  Missile  Defense 
Conference,  Nimble  Titan,  Joint  Deployable  Integrated  Air  and 
Missile  Defense,  Integrated  Air  and  Missile  Defense  Center 
Senior  Level  Seminar,  Huntsville  Wargame,  and  the  Missile 
Defense  Advisory  Committee  Wargame. 

•  To  support  operational  testing  beginning  in  FY13,  the 
MDA  completed  major  infrastructure  improvement  and 
modernization  efforts  at  the  Reagan  Test  Site  (RTS)  and 
on  Wake  Island.  At  RTS  these  include:  development  of 
Mission  Operations  Centers;  upgrade  of  power,  water, 
and  fuel  infrastructure  to  support  THAAD,  Patriot,  and 
AN/TPY-2  radar  sites  in  various  locations  on  the  Kwajalein 
Atoll;  development  of  housing  and  dining  facilities  for 
Soldiers  who  will  man  the  THAAD  and  Patriot  batteries; 

a  climate-controlled  storage  facility;  improvement  and 
modernization  of  housing  for  flight  test  personnel;  and 
installation  of  communications  infrastructure  to  support 
mission  data  and  voice  communications.  On  Wake  Island 
these  include:  refurbishment  of  the  50k  Rail  Launcher; 
improvement  and  upgrade  of  the  backup  power  systems; 
and  repair  and  improvement  of  the  existing  communications 
infrastructure. 

Assessment 

•  Many  of  the  models  and  simulations  used  in  the  ground 
tests  are  still  not  accredited  for  performance  assessment, 
thereby  limiting  quantitative  assessments  based  on  their 
results.  Re-baselining  the  IMTP  to  better  align  it  with  BMDS 
modeling  and  simulation  will  support  required  verification, 
validation,  and  accreditation. 

•  The  BMDS  defensive  capability  against  theater  threats 
increased  during  the  last  fiscal  year.  The  deployment  of  an 
AN/TPY-2  (FBM)  radar  to  Turkey  and  an  Aegis  BMD  ship 
to  the  Mediterranean  Sea  provided  an  interim  defensive 
capability  in  the  southeastern  Europe  theater,  which  was 
demonstrated  through  ground  testing  (although  these 
ground  tests  have  not  been  accredited  for  performance 
assessment).  Similar  ground  testing  provided  evidence  for 
an  initial  defensive  capability  for  the  Israeli  and  Middle 
Eastern  theaters  as  well.  During  FTI-01,  sensors  and  weapon 


systems  worked  together  successfully  to  engage  four  of  the 
five  targets  launched  during  the  test  scenario  contributing  to 
this  assessment.  Only  Aegis  BMD  was  unsuccessful  while 
engaging  a  short-range  ballistic  missile.  DOT&E  anticipates 
continued  increases  in  this  capability  over  time. 

•  BMDS  interoperability  increased  during  the  last  fiscal  year  to 
support  the  additional  defensive  capabilities  in  the  European, 
Israeli,  and  Middle  Eastern  theaters.  Ground  testing  (GT-04 
campaign  and  Fast  Eagle  Increment  1)  uncovered  some 
interoperability  and  command  and  control  deficiencies  that 
affected  track  processing,  situational  awareness,  and  battle 
management.  The  MDA  is  currently  investigating  these 
deficiencies. 

•  The  BMDS  capability  against  strategic  threats  has  not 
increased  because  the  GMD  program  continues  to  resolve 
deficiencies  with  its  Capability  Enhancement  II  (CE-II) 
Exo-atmospheric  Kill  Vehicle  (EKV).  As  a  result,  the  GMD 
program  did  not  conduct  a  flight  test  in  FY12. 

•  The  designated  Service  members  actively  participated  in 
system-level  BMDS  testing,  as  well  as  element-level  testing. 
They  perform  operational  roles  at  individual  element  levels 
through  major  Combatant  Command  levels  using  operational 
tactics,  techniques,  and  procedures. 

•  Although  the  LFG  has  developed  a  plan  of  action  to  address 
BMDS  lethality  data  voids,  the  MDA  has  made  little  progress 
in  retiring  lethality  data  needs. 

Recommendations 

•  Status  of  Previous  Recommendations.  Although  the  MDA 
has  made  progress  on  previous  recommendations,  the  FY08 
recommendation  regarding  the  BMDS  lethality  program  and 
the  FY09  recommendation  regarding  IMTP  execution  are  still 
valid.  The  F  Y 1 1  recommendation  regarding  flight  testing  to 
verify  root  causes  and  Failure  Review  Board  results  for  Aegis 
BMD  and  GMD  flight  tests  failures  has  been  partially  met  for 
the  Aegis  BMD  program,  and  is  still  being  addressed  for  the 
GMD  program. 

•  FY12  Recommendation. 

1.  The  MDA  should  continue  addressing  the  interoperability 
and  command  and  control  deficiencies  uncovered  during 
the  GT-04  test  campaign  and  FTI-01.  Resolution  of  these 
deficiencies  should  be  demonstrated  through  ground  and/or 
flight  testing. 
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Aegis  Ballistic  Missile  Defense  (Aegis  BMD) 


Executive  Summary 

•  The  Aegis  Ballistic  Missile  Defense  (BMD)  program 
completed  the  initial  phase  of  developmental  flight  testing 
of  the  Aegis  BMD  4.0. 1  and  4.0.2  defense  capabilities 
with  Standard  Missile-3  (SM-3)  Block  IB  interceptors,  and 
it  commenced  a  phase  of  combined  developmental  and 
operational  testing  (DT/OT)  for  the  4.0.1  and  4.0.2  system  and 
interceptor. 

•  The  Aegis  BMD  program  conducted  four  intercept  missions  in 
FY12;  three  were  successful  and  one  failed. 

•  Although  the  program  completed  the  FOT&E  phase  for  the 
Aegis  BMD  3.6.1  system  in  FY1 1,  the  system  continued 

to  take  part  in  BMDS-level  tests  related  to  Phase  1  of  the 
European  Phased  Adaptive  Approach  (EPAA)  and  other 
system-level  deployments. 

•  Aegis  BMD  continued  to  improve  interoperability  with  other 
Ballistic  Missile  Defense  System  (BMDS)  elements  and 
sensors  during  flight  and  ground  testing  in  FY12. 

•  Flardware-in-the-loop  (FIW1L)  ground  testing  demonstrated 
potential  Aegis  BMD  capability  to  contribute  to  theater-level 
defense  missions  spanning  a  range  of  ballistic  missile  defense 
scenarios. 

System 

•  Aegis  BMD  is  a  sea-based  missile  defense  system  that 
employs  the  multi-mission  shipboard  Aegis  Weapon  System, 
with  new  radar  and  missile  capabilities  to  engage  ballistic 
missile  threats.  Capabilities  of  Aegis  BMD  include: 

-  Computer  program  modifications  to  the  AN/SPY-1  radar, 
which  allow  long-range  surveillance  and  track  (LRS&T)  of 
ballistic  missiles  of  all  ranges. 

-  A  modified  Aegis  Vertical  Launch  System,  which  stores 
and  fires  SM-3  Block  1A  and  Block  IB  interceptors  (on 
select  ships),  and  modified  SM-2  Block  IV  interceptors  (on 
select  ships). 

-  SM-3  Block  IA  and  Block  IB  interceptors,  which  use  a 
maneuverable  kinetic  warhead  to  accomplish  midcourse 
engagements  of  short-,  medium-,  and  intermediate-range 
ballistic  missiles. 

-  Modified  SM-2  Block  IV  interceptors,  which  provide 
terminal  engagement  capability  against  short-range 
ballistic  missiles. 


•  Aegis  BMD  is  capable  of  performing  autonomous  missile 
defense  operations,  operations  that  exploit  networked  sensor 
information,  and  can  send  or  receive  cues  to  or  from  other 
BMDS  sensors  through  tactical  datalinks. 

Mission 

The  Navy  can  accomplish  three  missile  defense-related  missions 

using  Aegis  BMD: 

•  Defend  deployed  forces  and  allies  from  short-  to 
intermediate -range  theater  ballistic  missile  threats 

•  Provide  forward-deployed  radar  capabilities  to  enhance 
defense  against  ballistic  missile  threats  of  all  ranges  by 
sending  cues  or  target  track  data  to  other  elements  of  the 
BMDS 

•  Provide  all  short-  to  long-range  ballistic  missile  threat 
data  to  the  Command,  Control,  Battle  Management,  and 
Communications  (C2BMC)  system  for  dissemination  to 
Combatant  Commanders’  headquarters  to  ensure  situational 
awareness 

Major  Contractors 

•  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Moorestown,  New  Jersey 

•  Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  In  accordance  with  the  DOT&E -approved  Integrated  Master 
Test  Plan,  the  Aegis  BMD  program  completed  the  initial  phase 
of  developmental  flight  testing  of  the  Aegis  BMD  4.0. 1  and 
4.0.2  defense  capabilities  with  SM-3  Block  IB  interceptors, 


and  it  commenced  a  phase  of  combined  DT/OT  for  the  4.0.1 
and  4.0.2  system  and  interceptor. 

•  Although  the  program  completed  FOT&E  for  the  Aegis 
BMD  3.6.1  system  in  FY11,  the  program  continued  to  use 
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the  3.6.1  system  in  BMDS-level  tests  related  to  Phase  1  of  the 
EPAA  and  other  system-level  deployments. 

•  The  Aegis  BMD  program  conducted  four  intercept  missions  in 
FY12;  three  were  successful  and  one  failed: 

-  During  Flight  Test  SM-16  (FTM-16)  Event  2A  in 
May  2012,  an  Aegis  BMD  4.0.1  cruiser  intercepted  a 
short-range  non-separating  ballistic  missile  target  with 
an  SM-3  Block  IB  interceptor.  The  FTM-16  Event  2A 
engagement  was  the  first  intercept  of  a  ballistic  missile 
with  the  SM-3  Block  IB  interceptor  and  Aegis  BMD  4.0.1 
system. 

-  During  FTM-18  in  June  2012,  an  Aegis  BMD  cruiser  with 
4.0.1  software  intercepted  a  simple  separating  ballistic 
missile  target  with  an  SM-3  Block  IB  interceptor.  FTM-18 
was  the  second  successful  intercept  mission  conducted 
with  the  new  Aegis  BMD  4.0.1  system  with  an  SM-3 
Block  IB  interceptor,  and  the  first  combined  DT/OT  flight 
test  for  that  system. 

-  During  Flight  Test  lntegrated-01  (FTI-01)  in  October  2012, 
an  Aegis  BMD  3.6.1  ship  performed  a  near-simultaneous 
engagement  of  a  short-range  simple  separating  ballistic 
missile  target  with  an  SM-3  Block  IA  interceptor  and  an 
anti-air  warfare  target  with  an  SM-2  interceptor.  FTI-01 
was  the  first  integrated  flight  test  with  multiple  firing 
elements  (Aegis  BMD,  Terminal  High- Altitude  Area 
Defense  [THAAD],  and  Patriot)  engaging  multiple  ballistic 
missile  and  air-breathing  targets  in  a  realistic  BMDS-level 
architecture.  While  the  SM-3  Block  LA  interceptor  missed 
its  target,  the  SM-2  interceptor  achieved  a  successful 
intercept. 

•  In  FY12,  Aegis  BMD  participated  in  several  flight  and  ground 
tests  to  assess  Aegis  BMD  3.6.1  and  4.0.1  system  functionality 
and  interoperability  with  the  BMDS: 

-  Ground  Test  lntegrated-04d  (GTl-04d)  Part  2  in 
October  2011  tested  the  engagement  capabilities  of  existing 
missile  defense  systems  against  short-  and  medium-range 
ballistic  missiles  and  tested  system-level  sensor  resource 
management  and  tasking  in  an  HW1L  environment  in 
support  of  an  EPAA  Phase  1  assessment.  Participants 
included  Space-Based  Infrared  System/Defense  Support 
Program  (SB1RS/DSP),  Aegis  BMD  (laboratory  sites  with 
3.6.1  software),  AN/TPY-2  (Forward-Based  Mode  [FBM]), 
and  C2BMC. 

-  The  MDA  completed  GT1-04  (Israel)  in  November  2011, 
which  tested  the  interoperability  of  Israel’s  joint  BMDS.  It 
integrated  the  following  U.S.  and  Israeli  systems:  Aegis 
BMD  3.6.1,  C2BMC  AN/TPY-2  (FBM),  SBIRS/DSP, 
THAAD,  and  a  representation  of  the  Israeli  Arrow 
Weapons  System. 

-  During  Flight  Test  THAAD  (FTT)-12  in  October  2011,  an 
Aegis  BMD  laboratory  representation  was  utilized  to  assess 
the  capability  of  Aegis  BMD  3.6.1  to  conduct  simulated 
engagements  against  separating  and  non-separating 
short-range  ballistic  missiles  using  track  data  from  the 
Space  Tracking  and  Surveillance  System  (STSS). 


-  Ground  Test  Distributed-04d  (GTD-04d)  Part  2  in 
December  2011  tested  the  communication  architecture 
that  was  deployed  as  part  of  EPAA  Phase  1.  Participants 
included  a  laboratory  representation  of  Aegis  BMD  3.6. 1, 
and  AN/TPY-2  (FBM)  and  C2BMC  assets  in  a  distributed 
environment  using  operational  communication  systems  and 
operationally  representative  crews. 

-  GTD-04d  Part  3  in  December  2011  tested  interoperability 
and  engagement  capability  of  the  EPAA  Phase  1  system  in 
support  of  deployment.  Participants  included  Aegis  BMD 
(3.6.1  laboratory  representation),  AN/TPY-2  (FBM),  and 
C2BMC  in  a  distributed  environment  using  operational 
communication  systems  and  operationally  representative 
crews. 

-  Aegis  BMD  3.6.1  participated  in  the  FTM-15  System-level 
Post-Flight  Reconstruction  (SPFR)  in  December  2011, 
which  was  a  BMDS  HWlL-based  event  designed  to 
provide  data  in  support  of  modeling  and  simulation 
anchoring  efforts. 

-  Aegis  BMD  4.0. 1  laboratory  representations  participated 
in  Ground  Test  Other-O4e  (GTX-04e)  in  March  and 
April  2012.  GTX-04e  provided  the  necessary  HW1L 
architecture  for  testing  new  engagement  and  LRS&T 
capabilities  of  Aegis  BMD  4.0.1.  The  test  integrated 
C2BMC,  Ground-based  Midcourse  Defense,  AN/TPY-2 
(FBM),  Patriot,  Sea-Based  X-Band,  THAAD,  and  Aegis 
BMD  to  support  developmental  test  data  collection. 

-  Aegis  BMD  participated  in  the  Fast  Eagle  Increment  1 
HWIL  exercise  in  June  2012,  during  which  ballistic  missile 
defense  capabilities  were  explored  using  laboratory  assets 
for  Aegis  BMD  (3.6. 1),  AN/TPY-2  (FBM),  C2BMC, 
SBIRS/DSP,  and  Patriot,  with  U.S.  military  operators 
manning  the  systems.  These  tests  were  designed  to 
evaluate  the  capability  of  the  AN/TPY-2  (FBM)  and  U.S. 
Central  Command  (USCENTCOM)  C2BMC  to  augment 
the  existing  USCENTCOM  BMDS  capability. 

-  The  FTI-01  System  Pre-Mission  Test  (SPMT)  in  July  2012 
explored  integrated  engagement  capability  for  Aegis 
BMD,  THAAD,  and  Patriot  in  an  operationally  relevant 
architecture  using  HWIL  assets.  Aegis  BMD  3.6.1 
demonstrated  simultaneous  ship  self-defense  and  ballistic 
missile  defense  as  part  of  the  test. 

-  Fast  Eagle  Increment  1  Distributed  in  August  2012 
assessed  the  capability  of  AN/TPY-2  (FBM)  and  C2BMC 
to  augment  a  theater-regional  defense  architecture  with 
Aegis  BMD  (3.6. 1),  THAAD,  and  Patriot  fire  units.  U.S. 
military  operators  manned  deployed  tactical  BMDS  assets 
in  the  test. 

-  Aegis  BMD  3.6.1  participated  in  the  Fast  Eagle 
Increment  1+  HWIL  exercise  in  September  2012. 
Laboratory  representations  of  Aegis  BMD,  AN/TPY-2 
(FBM),  C2BMC,  SBIRS/DSP,  and  Patriot  explored 
ballistic  missile  defense  capabilities  in  a  theater  regional 
environment.  U.S.  military  operators  manned  the  systems. 
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Assessment 

•  In  FY12,  Aegis  BMD  demonstrated  the  capability  to  perform 
end-to-end  engagements  against  non-separating  and  simple 
separating  short-range  ballistic  missiles  with  the  Aegis 
BMD  4.0  system  and  SM-3  Block  IB  interceptors. 

•  In  response  to  the  anomalous  behavior  observed  during  the 
SM-3  Block  1A  flyout  in  FTM-15  (April  2011),  the  program 
redesigned  a  component  in  the  third  stage  rocket  motor,  which 
is  common  to  both  the  Block  IA  and  Block  IB  interceptors. 
The  newly  redesigned  component  was  flown  in  FTM-18  and 
performed  successfully. 

•  The  failed  intercept  in  FTM-16  Event  2  (September  2011) 
is  currently  being  addressed  by  the  program.  The  program 
conducted  three  initial  ground  firing  tests  of  the  SM-3  third 
stage  rocket  motor  to  further  understand  the  FTM-16  anomaly. 
Subsequently,  the  program  conducted  three  ground  firings  of 
the  third  stage  rocket  motor  to  further  verify  that  it  functions 
properly  using  newly-adjusted  firing  parameters.  Two  more 
ground  firings  are  planned  before  the  end  of  the  calendar  year 
to  close-out  actions  from  the  FTM-16  failure  review  board. 

•  GT-04  series  ground  tests  in  early  FY12,  which  addressed 
EPAA  Phase  1,  showed  that  improvements  in  interoperability 
are  needed  between  the  various  elements  and  sensors  that  are 
part  of  the  EPAA  Phase  1  defense  architecture,  including  the 
Aegis  BMD  3.6.1  system  that  continues  to  take  part  in  these 
tests  after  completion  of  its  FOT&E. 

•  The  near-simultaneous  engagement  of  an  anti-air  warfare 
target  during  FTI-0 1  verified  ship  self-defense  capability 


while  conducting  a  ballistic  missile  engagement  even  though 
the  SM-3  Block  IA  interceptor  missed  its  target.  The  MDA  is 
investigating  the  cause  of  the  missed  intercept;  however,  their 
efforts  will  be  hindered  because  Kill  Weapon  telemetry  was 
lost  during  key  portions  of  the  engagement  flyout. 

•  No  LRS&T  events  are  planned  for  Aegis  BMD  4.0  until 
FTG-08.  Aegis  BMD  has  tested  that  capability  only  once 
during  a  flight  test  (FTG-06a  in  December  2010)  and  in  ground 
testing  to  date.  Further  live-target  testing  of  this  capability  is 
needed  to  allow  for  an  assessment. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  partially 
addressed  the  single  recommendation  from  FY11  when  it 
conducted  FTM-18  testing  with  the  redesigned  component 
in  the  SM-3  third  stage  rocket  motor  (to  address  the  FTM-15 
anomaly).  Flight  testing  to  demonstrate  the  correction  for  the 
FTM-16  Event  2  failure  has  not  yet  taken  place. 

•  FY12  Recommendations.  The  program  should: 

1.  Conduct  further  live-target  testing  of  the  Aegis  BMD  4.0.2 
LRS&T  capability  using  long-range  targets  to  provide 
additional  data  on  that  capability  for  the  Aegis  BMD  4.0.2 
system. 

2.  Engage  a  medium-range  target  before  the  Full-Rate 
Production  Decision  for  the  SM-3  Block  IB  interceptor  to 
support  an  assessment  of  midcourse  defense  capability. 
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Command,  Control,  Battle  Management, 
and  Communications  (C2BMC)  System 


Executive  Summary 

•  During  FY12,  Command,  Control,  Battle  Management,  and 
Communications  (C2BMC)  participated  in  three  ground 
tests,  three  flight  tests,  exercise  Fast  Eagle  Increment  1,  and 
the  Israeli  Arrow  Weapon  System  (AWS)  Block  4  System 
Verification  Flight  Test. 

•  C2BMC  Spiral  6.4  (S6.4)  software  demonstrated  simultaneous 
live  control  of  two  AN/TPY-2  radars  operating  in 
Forward-Based  Mode  (FBM)  in  Turkey  and  Israel  in  support 
of  the  European  Phased  Adaptive  Approach  (EPAA)  Phase  1 
deployment. 

•  C2BMC  demonstrated  interoperability  with  Aegis  Ballistic 
Missile  Defense  (BMD),  Patriot,  Terminal  High- Altitude  Area 
Defense  (THAAD),  the  Space-Based  Infrared  System/Defense 
Support  Program  (SBIRS/DSP),  and  the  AWS. 

•  The  Missile  Defense  Agency  (MDA)  included  the  Global 
Engagement  Manager  (GEM)  in  the  deployment  of  C2BMC 
S6.4  to  the  Combatant  Commands.  GEM  provides  greater 
automation  of  sensor  management  functions  and  improved 
track  processing  and  reporting. 

•  Even  with  the  deployment  of  GEM,  C2BMC  still  has  battle 
management  capabilities  limited  primarily  to  situational 
awareness,  radar  track  forwarding,  and  control  of  AN/TPY-2 
(FBM)  radars  until  at  least  FY17  when  the  MDA  tentatively 
plans  to  field  S8.2. 

System 

•  C2BMC  is  a  Combatant  Command’s  interface  to  the  fully 
integrated  Ballistic  Missile  Defense  System  (BMDS). 

•  More  than  70  C2BMC  workstations  are  fielded  at  U.S. 
Strategic,  Northern,  European,  Pacific,  and  Central 
Commands  (USSTRATCOM,  USNORTHCOM,  USEUCOM, 
USPACOM,  and  USCENTCOM);  numerous  Army  Air  and 
Missile  Defense  Commands;  Air  and  Space  Operations 
Centers;  and  other  supporting  warfighter  organizations.  The 
current  C2BMC  system  provides  situational  awareness  to 
Combatant  Commands  and  the  National  Command  Authority 
with  information  on  missile  events,  BMDS  status,  and  system 
coverage.  C2BMC  also  provides  upper  echelon  deliberate 
planning  at  the  Combatant  Command  and  component  level, 
permitting  a  federation  of  planners  across  the  BMDS.  BMDS 
elements  (Aegis  BMD,  Ground-based  Midcourse  Defense 
[GMD],  Patriot,  and  THAAD)  use  their  own  command, 
control,  battle  management  systems  and  mission  planning 
tools  for  stand  alone  engagements. 

•  C2BMC  S6.4  provides  command  and  control  for  multiple 
AN /TPY-2  (FBM)  radars  in  the  same  area  of  responsibility. 


S6.4  includes  the  deployment  of  GEM,  which  includes 
updated  sensor  management,  track  processing,  and  reporting. 

•  C2BMC  provides  track  forwarding  of  AN/TPY-2  (FBM)  and 
AN/SPY- 1  tracks  to  GMD.  Additionally,  it  provides  track 
forwarding  of  AN/TPY-2  (FBM)  tracks  for  THAAD  and 
Patriot  cueing  and  Aegis  BMD  engagement  support. 

•  C2BMC  S8.2,  although  not  fully  defined  by  the  MDA,  is 
intended  to  improve  and  expand  the  initial  S6.4  capabilities 
as  the  next  step  toward  integrated  sensor  management.  The 
MDA  plans  to  field  S8.2  not  earlier  than  FY17. 

Mission 

U.S.  Strategic,  Northern,  European,  Central,  and  Pacific 

Commands  currently  use  C2BMC  to  support  ballistic  missile 

defense  engagements.  Commanders  use  C2BMC  specifically  for: 

•  Deliberate  and  dynamic  planning 

•  Situational  awareness 

•  Track  management 

•  AN/TPY-2  (FBM)  sensor  management  and  control 

•  Engagement  monitoring 

•  Data  exchange  between  C2BMC  and  BMDS  elements 

•  Network  management 

Major  Contractor 

Lockheed  Martin  Information  Systems  and  Global 

Solutions  -  Gaithersburg,  Maryland 
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Activity 

•  The  MDA  conducted  testing  during  FY 12  in  accordance  with 
the  DOT&E-approved  Integrated  Master  Test  Plan. 

•  The  USEUCOM  C2BMC  upgrade  to  S6.4  was  completed  at 
the  end  of  20 1 1  as  part  of  the  EPAA  Phase  1  deployment.  The 
USCENTCOM  C2BMC  upgrade  to  S6.4  will  complete  with 
the  MDA  publication  of  a  Technical  Capability  Declaration 

in  4QFY12.  Apart  from  already  existing  C2BMC  roles  in 
providing  situational  awareness,  sensor  management  for 
acquisition,  basic  track  forwarding,  and  some  planning 
capability,  S6.4  introduced  the  GEM  suite  at  USPACOM 
(with  a  backup  at  Missile  Defense  Integration  and  Operations 
Center),  USEUCOM,  and  USCENTCOM.  C2BMC 
participated  in  the  Flight  Test  THAAD-12  (FTT-12)  IOT&E 
in  October  2011,  demonstrating  interoperability  with  THAAD 
over  Tactical  Datalink  16.  C2BMC  provided  situational 
awareness  and  status  of  the  BMDS  under  test. 

•  The  MDA  conducted  Ground  Test  Integrated-04d  (GTl-04d) 
Part  2  in  October  2011.  The  USEUCOM  warfighters  used 
C2BMC  to  control  two  AN/TPY-2  (FBM)  radars.  C2BMC 
also  provided  situational  awareness  and  demonstrated 
interoperability  with  Aegis  BMD,  Patriot,  Israeli  AWS,  and 
SBIRS/DSP. 

•  The  MDA  completed  GT1-04  (Israel)  in  November  2011. 
C2BMC  controlled  one  AN/TPY-2  (FBM)  radar  and 
demonstrated  interoperability  with  Aegis  BMD,  Patriot, 
THAAD,  SBIRS/DSP,  and  the  Israeli  AWS. 

•  In  December  2011,  C2BMC  participated  in  Ground  Test 
Distributed-04d  (GTD-04d)  Part  2.  An  AN/TPY-2  (FBM) 
radar  was  represented  in  the  test  architecture  and  controlled  by 
C2BMC.  Aegis  BMD  and  SBIRS/DSP  also  participated. 

•  In  December  2011,  the  MDA  conducted  GTD-04d  Part  3  in 
support  of  the  EPAA  Phase  1  assessment.  The  USEUCOM 
sensor  managers  used  C2BMC  to  command  and  control  two 
AN/TPY-2  (FBM)  radars  for  the  first  time  in  a  distributed 
ground  test.  C2BMC  demonstrated  interoperability  with  Aegis 
BMD  and  SBIRS/DSP. 

•  In  April  2012,  the  MDA  conducted  a  developmental  ground 
test,  Ground  Test  Focused-04e  (GTX-04e),  and  collected 
data  for  several  C2BMC  critical  factors  (referred  to  as 
Critical  Engagement  Conditions).  C2BMC  demonstrated 
interoperability  with  the  full  BMDS  architecture,  provided 
situational  awareness,  and  controlled  AN/TPY-2  (FBM)  radars 
in  USPACOM  and  USEUCOM  areas  of  responsibility  (one 
and  two  radars,  respectively). 

•  C2BMC  participated  in  the  Fast  Eagle  Increment  1 
hardware-in-the-loop  test  in  June  2012  and  distributed  test  in 
August  2012.  It  demonstrated  the  ability  to  provide  situational 
awareness  and  command  and  control  of  one  AN/TPY-2  (FBM) 
radar,  and  interoperability  with  other  USCENTCOM  BMD 
assets. 

•  In  October  2012,  C2BMC  managed  a  deployed  AN/TPY-2 
(FBM)  radar  from  which  it  forwarded  acquisition  cues  to 
Aegis  BMD  and  THAAD  during  the  MDA’s  combined 
developmental/operational  Flight  Test  Integrated  -  01 
(FTI-01). 


Assessment 

•  GEM  allows  for  automated  management  of  multiple 
AN/TPY-2  (FBM)  sensors  located  in  one  area  of 
responsibility.  It  also  provides  greater  automation  of  sensor 
management  functions  and  improved  track  processing  and 
reporting  while  requiring  less  operator  involvement  as 
compared  to  S6.2  software. 

•  C2BMC  has  limited  battle  management  capabilities  allowing 
Combatant  Command  sensor  managers  to  direct  AN/TPY-2 
(FBM)  radars  to  execute  focused  search  plans  or  respond  to 

a  precision  cue.  S6.4  demonstrated  command  and  control  of 
a  single  AN/TPY-2  (FBM)  radar  in  ground  and  flight  tests. 

S6.4  demonstrated  command  and  control  of  two  AN/TPY-2 
(FBM)  radars  in  both  a  hardware-in-a-loop  and  distributed  test 
environment,  but  not  in  a  flight  test  using  deployed  assets. 

•  During  the  GTX-04e  event,  in  which  the  MDA  tested 
S6.4  interactions  with  strategic  elements,  C2BMC  S6.4 
demonstrated  interoperability  with  strategic  BMDS  elements 
and  the  ability  to  provide  situational  awareness. 

•  The  MDA  tested  C2BMC  S6.4  interactions  with  theater 
elements  throughout  the  GT-04  and  Fast  Eagle  Increment  1 
ground  test  campaigns  in  FY11  and  FY12.  In  addition  to 
providing  situational  awareness,  C2BMC  S6.4  demonstrated 
interoperability  with  theater  BMDS  elements  and  command 
and  control  of  one  and  two  AN/TPY-2  (FBM)  radars 
contributing  to  the  defensive  capability  for  the  southeastern 
European,  Israeli,  and  Middle  Eastern  theaters. 

•  During  the  GT-04  and  Fast  Eagle  Increment  1  campaigns, 
the  MDA  identified  S6.4  interoperability  and  command  and 
control  deficiencies  that  affected  track  processing,  situational 
awareness,  and  battle  management.  The  MDA  is  currently 
testing  solutions  to  these  deficiencies.  Some  interoperability 
issues  are  outside  of  the  C2BMC  program’s  authority  to 
resolve,  and  will  require  MDA  and  Service  action. 

•  C2BMC  appeared  to  successfully  manage  and  forward  cues 
from  the  deployed  AN/TPY-2  (FBM)  supporting  FTI-01. 
Analysis  is  ongoing.  DOT&E  will  report  results  in  the  2012 
BMDS  report  to  Congress,  scheduled  for  publication  in 
February  2013. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  addressed 
eight  of  the  previous  nine  recommendations.  The  MDA 
continues  to  make  progress  on  the  one  outstanding  FY06 
recommendation  to  include  assessments  of  Information 
Assurance  during  BMDS-centric  C2BMC  testing. 

•  FY12  Recommendation. 

1 .  The  MDA  should  continue  to  address  the  C2BMC 
interoperability  and  command  and  control  deficiencies 
uncovered  during  the  GT-04  campaign.  Resolution  of  these 
deficiencies  should  be  demonstrated  through  ground  and/or 
flight  testing. 


286  C2BMC 


BALLISTIC  MISSILE  DEFENSE  SYSTEMS 


Ground-Based  Midcourse  Defense  (GMD) 


Executive  Summary 

•  Capability  Enhancement  II  (CE-II)  kill  vehicle  problems 
limited  advancement  of  the  Ground-based  Midcourse  Defense 
(GMD)  flight  test  program.  No  flight  tests  were  conducted 
during  FY12. 

•  The  Missile  Defense  Agency  (MDA)  did  make  progress 
on  GMD  return-to-intercept  by  conducting  a  ground  test 
campaign  consisting  of  1 1  electrical  and  mechanical  tests 
designed  to  further  characterize  CE-II  kill  vehicle  component 
capability  and  performance.  The  MDA  has  scheduled  a 
two-flight  test  series  in  FY13  designed  to  confirm  fixes  to  the 
CE-II  kill  vehicle. 

System 

GMD  is  a  Ballistic  Missile  Defense  System  (BMDS)  element 

that  counters  intermediate-range  and  intercontinental  ballistic 

missile  threats  to  the  U.S.  Homeland.  The  GMD  BMDS 

includes: 

•  Cobra  Dane  Upgrade  Radar  at  Eareckson  Air  Station  (Shemya 
Island),  Alaska 

•  Upgraded  Early  Warning  Radars  (UEWR)  at  Beale  AFB, 
California;  Royal  Air  Force  (RAF)  Fylingdales,  United 
Kingdom;  and  Thule  Air  Base,  Greenland 

•  Ground-based  Interceptor  (GB1)  missiles  at  Fort  Greely, 
Alaska,  and  Vandenberg  AFB,  California 

•  GMD  ground  system  including  GMD  Fire  Control  (GFC) 
nodes  at  Schriever  AFB,  Colorado,  and  Fort  Greely,  Alaska; 
Command  Launch  Equipment  (CLE)  at  Vandenberg  AFB, 
California,  and  Fort  Greely,  Alaska;  and  In-Flight  Interceptor 
Communication  System  Data  Terminals  (IDTs)  at  Vandenberg 
AFB,  California,  Fort  Greely,  Alaska,  and  Shemya  Island, 
Alaska 

•  GMD  secure  data  and  voice  communication  system  including 
long-haul  communications  using  the  Defense  Satellite 
Communication  System  (DSCS),  commercial  satellite 
communications,  and  fiber  optic  cable  (both  terrestrial  and 
submarine) 

•  External  interfaces  that  connect  to  Aegis  Ballistic  Missile 
Defense  (BMD);  North  American  Aerospace  Defense  -  U.S. 
Northern  Command  (NORAD-NORTHCOM)  Command 
Center  (N2C2)  and  Command,  Control,  Battle  Management, 
and  Communications  (C2BMC)  at  Peterson  AFB,  Colorado; 
Space  Based  Infrared  System/Defense  Support  Program 
(SBIRS/DSP)  at  Buckley  AFB,  Colorado;  and  AN/TPY-2 
(Forward-Based  Mode  [FBM])  radar  at  Shariki  Air  Base, 
Japan 


•  Sea-Based  X-Band  (SBX)  radar,  which  is  a  sea-based  mobile 
sensor  platform  primarily  for  use  as  a  test  asset,  but  can  be 
operationally  deployed  as  needed 

Mission 

Military  operators  for  the  U.S.  Army  Space  and  Missile  Defense 
Command/ Army  Forces  Strategic  Command  (the  Army  service 
component  to  U.S.  Strategic  Command)  will  use  the  GMD 
system  to  defend  the  U.S.  Homeland  against  intermediate-range 
and  intercontinental  ballistic  missile  attacks  using  its  weapon,  the 
GBI,  to  defeat  threat  missiles  during  the  midcourse  segment  of 
flight. 

Major  Contractors 

•  The  Boeing  Company,  Integrated  Defense  Systems,  Missile 
Defense  Systems  -  Huntsville,  Alabama 

•  Orbital  Sciences  Corporation  -  Chandler,  Arizona 

•  Raytheon  Missile  Systems  -  Tucson,  Arizona 

•  Northrop  Grumman  Information  Systems  -  Huntsville, 
Alabama 
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Activity 

•  The  MDA  conducted  a  GMD  return-to-intercept  ground 
test  campaign  (G-52-L)  during  2QFY 12.  It  consisted  of  1 1 
individual  electrical  and  mechanical  tests  designed  to  further 
characterize  CE-I1  kill  vehicle  component  capability  and 
performance. 

•  The  MDA  used  hardware  and  software  representations  of 
the  GFC  IDTs  and  GBIs  in  the  Ground  Test  Focused-04e 
(GTX-04e)  in  May  2012.  The  MDA  conducted  the 
GMD  portion  of  the  test  to  demonstrate  functionality, 
interoperability,  and  performance  of  the  GFC  software 
version  6B2.1. 

Assessment 

•  The  MDA  made  progress  on  GMD  return-to-intercept.  It 
redesigned  and  tested  Exo-atmospheric  Kill  Vehicle  (EKV) 
components  and  established  more  stringent  component  and 
manufacturing  process  requirements.  Analysis  and  ground  test 
results  indicate  the  previous  mission  failure  root  causes  have 
been  identified  and  fixed,  but  a  demonstration  flight  test  is 
required  to  validate  that  conclusion. 

-  The  MDA  has  scheduled  a  two-flight  test  series  designed 
to  validate  these  fixes.  The  GMD  Control  Test  Vehicle-01 
(GM  CTV-01)  has  been  scheduled  for  2QFY13  as 

an  interceptor-only  diagnostic  flight  test  to  further 
characterize  kill  vehicle  behavior  in  a  representative  flight 
environment.  The  Flight  Test  GMD-06b  (FTG-06b)  has 
been  scheduled  for  3QFY13  as  a  repeat  of  the  FTG-06a 
flight  test. 

-  These  current  test  windows  represent  the  latest  in  a  series 
of  incremental  slips  over  FY12.  At  the  beginning  of  FY12, 


GM  CTV-01  was  scheduled  to  be  flown  in  the  middle 
of  3QFY12  and  FTG-06b  was  scheduled  for  the  middle 
of  4QFY12.  Major  causes  of  the  slips  were  additional 
analysis  time  needed  by  the  Failure  Review  Board, 
developmental  issues  with  new  inertial  measurement  unit 
firmware  and  isolation  cradle  hardware,  and  component 
manufacturing  and  quality  concerns. 

•  GTX-04e  demonstrated  the  capability  to  provide  dynamic 
positioning  updates  to  the  GMD  IDT  onboard  the  SBX 
radar  platform;  the  MDA  had  previously  demonstrated 
this  capability  for  the  IDT  at  Fort  Greely,  Alaska,  and 
Vandenberg  AFB,  California.  Further,  the  MDA  confirmed 
the  transmission  and  data  flow  of  a  new  message  set  from  the 
AN/TPY-2  (FBM)  radar  to  the  new  GFC  software  via  the 
C2BMC. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  has  started 
but  not  completed  the  FY 1 1  recommendation  to  repeat  the 
FTG-06a  mission  to  verify  root  causes,  Failure  Review 
Board  results,  and  permanent  fixes  for  the  deficiencies  found 
during  the  flight  test.  They  have  identified  root  cause  issues, 
implemented  solutions,  and  scheduled  a  two-flight  test  series 
in  FY13  designed  to  demonstrate  GMD  return-to-intercept. 
The  FY07  recommendation  to  re-examine  the  GMD-specific 
lethality  simulation  needs  has  been  transferred  to  the  BMDS 
system-level  assessment  since  it  applies  to  all  elements. 

•  FY12  Recommendations.  None. 
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Terminal  High-Altitude  Area  Defense  (THAAD) 


Executive  Summary 

•  The  Terminal  High- Altitude  Area  Defense  (THAAD)  system 
intercepted  two  short-range  targets  nearly  simultaneously 

in  October  2011.  The  program  completed  this  multiple 
simultaneous  intercept  as  part  of  an  IOT&E,  which  included 
a  full  battle  sequence  from  planning  through  intercept  under 
operationally  realistic  conditions.  DOT&E  concluded  that 
the  IOT&E  demonstrated  THAAD  is  operationally  effective, 
operationally  suitable,  and  survivable  against  the  threats  and  in 
the  environments  tested. 

•  The  THAAD  system  successfully  intercepted  a  medium-range 
ballistic  missile  target  for  the  first  time  in  October  2012. 

•  All  planned  THAAD  Build  1.0  capabilities  have  not  yet 
been  demonstrated.  The  most  significant  example  is  that  the 
performance  of  the  system  using  the  radar  advanced  algorithm 
against  a  complex  target  has  not  been  scheduled  for  test  until 
FY14.  The  algorithm  has  been  implemented  in  the  operational 
software,  but  THAAD  flight  test  profiles  prior  to  FY14  are  not 
expected  to  trigger  demonstration  of  it. 

•  Redesign  and  retesting  of  a  number  of  components  are 
required  to  address  all  of  the  Army  materiel  release  conditions 
imposed  before  full  materiel  release  can  be  granted.  In 
particular,  many  reliability  improvements  are  required  to  meet 
Army  requirements  with  confidence. 

System 

•  The  THAAD  ballistic  missile  defense  system  consists  of  five 
major  components: 

-  Missiles 

-  Launchers 

-  Radars  (designated  AN/TPY-2  (TM)  for  Terminal  Mode) 

-  THAAD  Fire  Control  and  Communications  (TFCC) 

-  Unique  THAAD  support  equipment 

•  THAAD  can  accept  target  cues  for  acquisition  from  the  Aegis 
Ballistic  Missile  Defense  (BMD),  satellites,  and  other  external 
theater  sensors  and  command  and  control  systems. 

•  THAAD  is  intended  to  complement  the  lower-tier  Patriot 
system  and  the  upper-tier  Aegis  BMD  system. 


Mission 

U.S.  Strategic  Command  intends  to  deploy  and  employ 
THAAD,  a  rapid  response  weapon  system,  to  protect  critical 
assets  worldwide.  Commanders  will  use  the  THAAD  kill 
vehicle  to  intercept  an  incoming  threat  ballistic  missile  in  the 
endo-atmosphere  or  exo-atmosphere,  limiting  the  effects  of 
weapons  of  mass  destruction  on  battlefield  troops  and  civilian 
populations. 

Major  Contractors 

•  Lockheed  Martin  Missile  and  Fire  Control  -  Dallas,  Texas 

•  Lockheed  Martin  Space  Systems  Company  -  Sunnyvale, 
California 

•  Raytheon  Integrated  Defense  Systems  -  Tewksbury, 
Massachusetts 


Activity 

•  Flight  Test  THAAD  Interceptor- 1 2  (FTT-12)  IOT&E  occurred 
in  October  2011.  The  test  was  a  multiple  simultaneous 
engagement  of  two  short-range  targets.  This  test  supported 
materiel  release  of  the  first  two  THAAD  batteries  and  future 
Beyond  Low-Rate  Initial  Production  decisions.  The  THAAD 
battery  performed  battle  planning,  overseas  deployment, 
emplacement,  and  mission  operations  under  operationally 
realistic  conditions  within  the  constraints  of  test  range  safety. 


The  THAAD  battery  also  conducted  additional  simulated 
intercept  events  against  a  raid,  defeating  threats  generated  by 
the  Simulation  Over  Live  Driver  (SOLD). 

•  The  combined  developmental/operational  Flight  Test 

Integrated-0 1  (FTI-01)  in  October  2012  included  a  THAAD 
engagement  against  a  medium-range  target  for  the  first  time. 
The  test  evaluated  interoperability  between  THAAD;  Aegis 
BMD;  Patriot;  Command,  Control,  Battle  Management,  and 


THAAD  289 


BALLISTIC  MISSILE  DEFENSE  SYSTEMS 


Communications  (C2BMC);  and  AN/TPY-2  Forward-Based 
Mode  (FBM)  elements  with  multiple  live  targets. 

•  Ground  Test  lntegrated-04  Israel  (GT1-04  1SR)  in 
November  2011,  Ground  Test  Other-O4e  (GTX-04e)  in 
April  2012,  Fast  Eagle  Increment  1  Hardware-ln-The-Loop 
(FIW1L)  in  June  2012,  and  GTI-04e  in  November  2012 
included  laboratory  F1WIL  representations  of  TF1AAD. 
Interoperability,  engagement  coordination  between  the  theater 
elements,  and  engagement  capabilities  against  short-  and 
medium-range  ballistic  missiles  were  tested  using  BMDS 
configurations  that  are  deployed  or  nearing  deployment. 

•  The  MDA  conducted  testing  during  FY 12  in  accordance  with 
the  DOT&E-approved  Integrated  Master  Test  Plan. 

Assessment 

•  The  TF1AAD  and  AN/TPY-2  Radar  systems  performed  a 
successful  engagement  of  two  targets  during  the  FTT-12  test. 
The  classified  DOT&E  February  2012  THAAD  and  AN/TPY-2 
Radar  Operational  and  Live  Fire  Test  and  Evaluation  Report 
concluded  the  following: 

-  THAAD  is  operationally  effective  against  short-range 
ballistic  missile  threats  of  the  types  tested  to  date.  It  has 
not  been  demonstrated  against  medium-range  threats. 
However,  empirical  data  from  short-range  flight  testing, 
ground  testing,  and  analyses  indicate  THAAD  likely  has 
capability  against  medium-range  threat  missiles. 

-  THAAD  is  operationally  suitable.  Nevertheless, 
examination  of  reliability  data,  ground  test  results, 
challenges  encountered  during  testing,  and  Soldier 
feedback  indicate  that  THAAD  has  suitability-related 
limitations.  Adequate  availability  and  maintainability 
were  demonstrated,  but  testing  identified  maintenance 
shortfalls.  Different  failure  modes  were  seen  in  two  tests 
creating  uncertainty  in  the  Mean  Time  Between  System 
Abort.  Improvements  are  also  needed  in  deployability, 
manpower  and  training,  human  factors  engineering,  and 
interoperability. 

-  THAAD  is  survivable  in  chemical,  biological,  radiological, 
and  external  electromagnetic  environments.  It  has  not  been 
tested  in  electronic  warfare  environments. 

•  Conditional  Materiel  Release  of  the  first  two  THAAD  batteries 
in  February  2012  included  39  conditions  that  need  to  be 
resolved  before  a  full  materiel  release  could  be  granted.  The 
THAAD  Project  Office  and  the  Army  have  begun  to  address 
these  conditions  including  verification  testing  of  the  thermally 
initiated  venting  system  on  the  interceptor,  electrical  stress 
testing  of  the  optical  block  in  the  interceptor  flight  sequencing 
assembly,  and  validation  and  verification  demonstrations 

of  changes  and  updates  to  the  technical  manuals.  Four 


conditions  (equipment  grounding,  air  load  certification,  spares 
transport  shelter,  and  the  Surface  Deployment  and  Distribution 
Command-Transportation  Engineering  Agency  transportability 
certification)  have  been  closed.  Analyses  of  data  collected 
during  the  FTI-01  test  are  ongoing,  which  potentially  will 
close  eight  additional  conditions.  Fixes  and  testing  of 
remaining  conditions  are  scheduled  through  20 1 7. 

•  Initial  assessment  from  the  FTI-0 1  test  mission  data 
indicated  that  the  THAAD  system  successfully  intercepted  a 
medium-range  ballistic  missile  target.  The  interoperability 
assessment  between  THAAD  and  other  elements  based  on 
FTI-01  test  data  is  ongoing. 

•  Ground  tests  utilizing  HWIL  representations  of  THAAD 
demonstrated  interoperability  and  engagement  coordination 
between  THAAD  and  other  theater  elements  revealing 
problems  that  need  to  be  addressed  for  multi-element 
coordination. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA 
has  satisfactorily  addressed  all  previous  THAAD 
recommendations . 

•  FY12  Recommendations.  The  MDA  and  the  Army  have 
begun  to  address  the  22  THAAD  recommendations 
contained  in  the  classified  DOT&E  February  2012  THAAD 
and  AN/TPY-2  Radar  Operational  and  Live  Fire  Test  and 
Evaluation  Report.  Fifteen  of  the  recommendations  align 
directly  with  the  Army  materiel  release  conditions,  which  are 
being  addressed  through  a  corrective  action  plan  agreed  to  by 
the  THAAD  Project  Office  and  the  Army.  Of  the  remaining 
seven  recommendations,  three  are  classified  (Effectiveness  #2, 
Effectiveness  #5,  and  Survivability  #4).  The  four  remaining 
unclassified  recommendations  are: 

1.  The  MDA  and  the  Army  should  reassess  the  required  spares 
and  tools  (including  their  quantities)  that  should  be  on 

site  with  the  battery  based  on  all  available  reliability  and 
maintainability  data  (Suitability  #5). 

2.  The  MDA  and  the  Army  should  define  duties  related 
to  THAAD  at  the  brigade  level.  Until  a  battalion  is 
established  for  THAAD,  it  should  also  define  duties  and 
training  for  THAAD  battery  personnel  on  any  required 
battalion-level  duties  (Suitability  #10). 

3.  The  MDA  and  the  Army  should  implement  equipment 
redesigns  and  modifications  identified  during  natural 
environment  testing  to  prevent  problems  seen  in  testing 
(Suitability  #11). 

4.  The  MDA  and  the  Army  should  conduct  electronic  warfare 
testing  and  analysis  (Survivability  #3). 
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Sensors 


Executive  Summary 

•  Ballistic  Missile  Defense  System  (BMDS)  sensors  participated 
in  five  ground  tests,  three  flight  tests,  and  exercise  Fast  Eagle 
Increment  1  testing  during  the  reporting  period. 

•  Although  the  Missile  Defense  Agency  (MDA)  placed  the 
Sea-Based  X-band  (SBX)  radar  into  a  limited  test  support  and 
standby  operational  status,  the  MDA  continued  to  include  it  in 
ground  tests  when  appropriate. 

•  Accreditation  of  each  of  the  sensor  models  for  use  in 
performance  assessments  continues  to  progress  but  is 
incomplete. 

•  The  BMDS  Operational  Test  Agency  Team  will  be  unable 
to  accredit  the  Cobra  Dane  radar  model  until  after  the  MDA 
completes  a  3QFY15  flight  test  involving  the  radar. 

System 

The  BMDS  sensors  are  systems  that  provide  real-time  ballistic 

missile  threat  data  to  the  BMDS.  The  data  are  used  to  counter 


infrared  sensors  operated  by 
the  Air  Force  with  an  external 
interface  to  the  BMDS  located 
at  Buckley  AFB,  Colorado 
•  Fixed  site,  fixed  orientation, 
phased  array  radars 

-  Cobra  Dane  Upgrade 
(CDU)  Radar,  an  L-band 
radar  (one  radar  face 
that  provides  120-degree 
azimuth  field  of  view) 
operated  by  the  Air  Force 
and  located  at  Eareckson 
Air  Station  (Shemya 
Island),  Alaska 

-  Upgraded  Early  Warning 
Radars  (UEWRs),  ultra 
high  frequency  radars 
operated  by  the  Air  Force 
and  located  at  Beale  AFB, 
California  (two  radar  faces 
that  provide  240-degree 
azimuth  field  of  view); 
Fylingdales,  United 
Kingdom  (three  radar  faces 


ballistic  missile  attacks.  These  sensor  systems  are  operated 
by  the  Army,  Navy,  Air  Force,  and  the  MDA,  and  include  a 
satellite-based,  infrared  sensor  system  and  five  phased  array  radar 
system  types.  The  sensor  systems  are: 

•  Space  Based  Infrared 
System/Defense  Support 
Program  (SBIRS/DSP),  a 
satellite  constellation  of 


Cobra  Dane 


UEWR 


that  provide  360-degree  azimuth  field  of  view);  and  Thule, 
Greenland  (two  radar  faces  that  provide  240-degree  azimuth 
field  of  view).  [The  MDA  and  Air  Force  Space  Command 
awarded  a  contract  in  July  2012  for  the  upgrade  of  the  Early 
Warning  Radar  (EWR)  at  Clear  Air  Force  Station,  Alaska. 
The  contract  also  included  an  option  for  the  upgrade  of  the 
EWR  at  Cape  Cod  Air  Force  Station,  Massachusetts.] 

•  Mobile  platform,  variable  orientation,  phased  array  radars 

-  AN/TPY-2  (Forward-Based 
Mode  [FBM])  Radars, 

X-band  radars  (one 
radar  face  that  provides 
120-degree  azimuth  field 
of  view)  operated  by 
the  Army  and  located  at 
Shariki  Air  Base,  Japan, 
and  sites  in  Israel  and 
Turkey 

-  Aegis  Ballistic  Missile 
Defense  (Aegis  BMD) 

AN/SPY- 1  Long-Range 
Search  and  Track 
(LRS&T)  Radars,  S-band 
radars  (four  radar  faces 
that  provide  360-degree 
azimuth  field  of  view) 
operated  by  the  Navy 
and  located  aboard  Aegis 
BMD-capable  cruisers  and 
destroyers 

-  Sea-Based  X-Band  (SBX) 

Radar,  an  X-band  radar 
operated  by  BMDS  and 
located  aboard  a  twin 


hulled,  semi-submersible, 
self  propelled,  ocean  going 
platform  (primarily  a  test 
asset  that  can  be  operationally  deployed  as  needed) 


Mission 

Military  operators  for  the  U.S.  Strategic  Command,  U.S.  Northern 

Command,  U.S.  European  Command,  U.S.  Pacific  Command,  and 

U.S.  Central  Command  will  use  the  BMDS  sensors  to: 

•  Detect,  track,  and  classify  ballistic  missile  threats  that  target  the 
United  States,  U.S.  allies,  and  U.S.  friends 

•  Provide  data  for  situational  awareness  and  battle  management 
to  the  BMDS  Command,  Control,  Battle  Management,  and 
Communications  (C2BMC) 

•  Provide  data  that  support  engagement  of  ballistic  missile  threats 
by  ballistic  missile  defense  systems 
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Major  Contractors 

Aegis  AN/SPY-1 

•  Lockheed  Martin  -  Moorestown,  New  Jersey 

AN/TPY-2 

•  Raytheon  Company,  Integrated  Defense  Systems  -  Woburn, 
Massachusetts 

CDU 

•  Raytheon  Company,  Integrated  Defense  Systems  -  Woburn, 
Massachusetts 

SBIRS 

•  Lockheed  Martin  Space  Systems  Company  -  Sunnyvale, 
California 


SBX 

•  Raytheon  Company,  Integrated  Defense  Systems  -  Woburn, 
Massachusetts 

UEWRs/EWRs 

•  Thule,  Beale,  and  Fylingdales 

-  Exelis  -  Colorado  Springs,  Colorado 

•  Clear  and  Cape  Cod 

-  Raytheon  Company,  Integrated  Defense 
Systems  -  Woburn,  Massachusetts 


Activity 

•  The  MDA  conducted  testing  during  FY 12  in  accordance  with 
the  DOT&E-approved  Integrated  Master  Test  Plan. 

•  The  MDA  used  an  AN/TPY-2  (FBM)  radar,  SBX  radar,  Cobra 
Dane  radar,  and  the  UEWRs  at  Beale,  Fylingdales,  and  Thule 
in  a  BMDS-level  satellite  test  campaign.  Using  these  radars, 
the  MDA  tracked  selected  satellites  to  collect  data  that  the 
MDA  and  the  BMDS  Operational  Test  Agency  Team  use  to 
verify,  validate,  and  accredit  the  radar  models  and  simulations 
used  to  assess  BMDS  performance. 

Aegis  BMD  Radar 

•  The  MDA  used  a  hardware-in-the-loop  (HWIL) 
representation  of  the  Aegis  BMD  radar  in  its  long-range 
surveillance  and  track  mode  in  Ground  Test  Focused-04e 
(GTX-04e)  in  April  2012. 

AN/TPY-2  (FBM)  Radar 

•  The  AN/TPY-2  (FBM)  radar  participated  in  Flight  Test 
Integrated-0 1  (FTI-01)  in  October  2012.  In  FTI-01,  the 
AN/TPY-2  (FBM)  radar  provided  up-range  track  data  to 
C2BMC  for  processing,  down-select,  and  forwarding  of 
tracks  to  Aegis  BMD,  Patriot,  and  Terminal  High-Altitude 
Area  Defense  (THAAD). 

•  In  1QFY12,  deployed  AN/TPY-2  (FBM)  radars  participated 
in  Ground  Tests  Distributed  (GTDs)  to  demonstrate  Phase  1 
European  Phased  Adaptive  Approach  (EPAA)  capability: 

-  GTD-04d  (Part  2)  provided  the  data  required  for  a 
Technical  Capability  Declaration  of  the  EPAA  Phase  1 
radar. 

-  GTD-04d  (Part  3)  completed  the  EPAA  Phase  1 
Capability  demonstration  including  C2BMC  control  of 
multiple  deployed  AN/TPY-2  radars. 

•  The  AN/TPY-2  (FBM)  radar  participated  in  the 
BMDS-level  distributed  ground  test  Fast  Eagle  Increment  1 
in  August  2012.  The  MDA  used  this  testing  to  evaluate  the 
capability  of  the  AN/TPY-2  (FBM)  radar  and  the  associated 
C2BMC  to  augment  existing  Combatant  Command  BMDS 
capability. 

•  The  MDA  used  an  HWIL  representation  of  the  AN  /  TP Y-2 
(FBM)  radar  in  these  BMDS-level  ground  tests:  Ground 
Test  Integrated-04d  (GTl-04d)  Part  2  (October  2011), 


GTI-04  (Israel  [ISR])  (November  2011),  GTX-04e 
(April  2012),  and  Fast  Eagle  Increment  1  (June  2012  and 
September  2012). 

•  AN/TPY-2  (FBM)  radars  participated  in  BMDS  Integrated 
Sensor  Demonstration  -  Space  Situational  Awareness 
(BISD-SSA),  a  Joint  Chiefs  of  Staff  sponsored  test  with 
C2BMC  and  Air  Force  Space  Command.  This  test 
demonstrated  the  ability  of  AN/TPY-2  (FBM)  radars  to 
acquire  and  track  space  objects.  The  Joint  Space  Operations 
Center  (JSPOC)  sent  tasking  messages  to  C2BMC,  which 
converted  the  messages  into  radar  task  commands  that  the 
radars  executed.  Data  were  collected  and  sent  to  JSPOC  in 
real  time,  which  then  published  the  data  for  authorized  user 
access. 

Cobra  Dane  Radar 

•  HWIL  representations  of  the  Cobra  Dane  radar  participated 
in  the  BMDS-level  ground  test  GTX-04e  in  April  2012. 

•  In  F Y 12,  the  Air  Force  used  the  Cobra  Dane  radar  to 
observe  targets  of  opportunity.  The  U.S.  Space  Command 
also  used  the  Cobra  Dane  radar  as  a  contributory  sensor  to 
the  Space  Surveillance  Network  to  track  orbital  debris  and 
active  satellites. 

SBIRS/DSP  System 

•  During  FY12,  the  Air  Force  used  the  SBIRS/DSP  system  to 
observe  domestic  and  foreign  launch  events,  provide  launch 
event  data  to  the  operational  BMDS,  and  participate  in  the 
intercept  flight  tests:  Flight  Test  Standard  Missile-3-16 
(FTM-16)  Event  2a,  FTM-18,  and  FTI-01. 

•  A  digital  representation  of  the  SBIRS/DSP  system 
participated  in  the  BMDS-level  ground  tests:  GTl-04d 
Part  2  (October  2011),  GTI-04  (ISR)  (November  2011), 
GTX-04e  (April  2012),  and  Fast  Eagle  Increment  1 
(August  2012). 

SBX  Radar 

•  An  HWIL  representation  of  the  SBX  radar  participated  in 
the  BMDS-level  ground  test  GTX-04e  in  April  2012. 

•  Both  an  HWIL  and  digital  representations  (simulations) 
of  the  SBX  radar  participated  in  pre-mission  testing  for 
the  upcoming  Ground-based  Midcourse  Defense  (GMD) 
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intercept  Flight  Test  Ground-based  Interceptor-06B 
(FTG-06b). 

•  The  MDA  utilized  Glory  Trip-203,  a  Minuteman  III 
intercontinental  ballistic  missile  test,  to  demonstrate  the 
ability  of  SBX  to  exercise  software  build  2. 2. 1.2. 5  in  a 
live  track  mission,  collecting  advanced  discrimination 
algorithm  data  for  future  use,  verifying  algorithms,  and 
supporting  analysis/risk  reduction  for  upcoming  tests,  such 
as  FTG-06b. 

•  The  MDA  placed  SBX  in  a  limited  test  support  status. 

The  SBX  can  be  reactivated  based  on  warning  of  a  threat 
presented  to  the  United  States,  and  for  BMDS  flight  testing. 

UEWR/EWR  -  Beale,  Thule,  Fylingdales,  Clear,  and  Cape  Cod 

•  HWIL  representations  of  the  UEWRs  at  Beale,  Fylingdales, 
and  Thule  participated  in  the  BMDS-level  ground  test 
GTX-04e  in  April  2012. 

•  In  FY12,  the  U.S.  Air  Force  used  the  Beale,  Fylingdales, 
and  Thule  UEWRs,  and  Clear  and  Cape  Cod  EWRs,  to 
observe  targets  of  opportunity.  U.S.  Space  Command 
also  used  these  radars  as  collateral  sensors  to  the  Space 
Surveillance  Network  to  track  orbital  debris  and  active 
satellites. 

Assessment 

•  The  MDA  has  gained  significant  operational  experience  with 
each  of  the  BMDS  sensors  since  the  completion  of  sensor 
upgrade  and  development  programs. 

•  The  MDA  and  the  BMDS  Operational  Test  Agency  Team, 
however,  have  not  fully  accredited  models  and  simulations 
of  the  BMDS  sensors  for  performance  assessment. 
Representations  of  the  AN/TPY-2  (FBM)  radar,  the  SBX 
radar,  and  the  UEWR  have  been  accredited  for  limited  uses. 
Representations  of  the  Aegis  BMD  radar,  the  Cobra  Dane 
radar,  and  the  SBIRS/DSP  system  have  not  been  accredited. 
The  MDA  is  analyzing  the  radar  performance  data  that  were 
collected  in  the  FY12  satellite  tracking  campaign  for  its  use 
toward  accreditation  of  multiple  radar  models  and  simulations. 

•  Starting  in  FY13,  the  U.S.  Air  Force  will  have  the 
responsibility  for  the  sustainment  of  the  Cobra  Dane  radar  and 
the  UEWRs.  The  U.S.  Air  Force  does  not  currently  gather  all 
of  the  operational  data  required  for  assessing  suitability. 

Aegis  BMD  Radar 

•  In  GTX-04e,  and  in  prior  ground  tests,  the  MDA 
demonstrated  a  capability  of  the  Aegis  BMD  radar  (in 
its  LRS&T  mode)  to  support  GMD  engagement  of 
intermediate-range  and  intercontinental  ballistic  missile 
threats. 

•  The  Aegis  BMD  radar  provided  data  that  enabled  the  GMD 
system  to  generate  sensor  cueing  and  missile  engagement 
plans.  However,  a  quantitative  assessment  of  the  Aegis 
BMD  radar  LRS&T  mode  performance  was  not  possible 
since  the  MDA  used  some  models  and  simulations 
(including  the  intermediate -range  and  intercontinental 
ballistic  missile  threat  models)  that  were  not  accredited  for 
performance  assessment. 


•  The  MDA  has  not  conducted  a  BMDS  intercept  flight  test 
that  uses  the  Aegis  BMD  LRS&T  radar  data  in  real-time 
as  the  primary  data  source  for  GMD  engagement  planning. 
The  MDA  currently  plans  to  conduct  this  test  during 
3QFY14. 

AN/TPY-2  (FBM)  Radar 

•  In  FTI-0 1,  the  MDA  demonstrated  AN/TPY-2  (FBM)  radar 
capability  to  provide  up-range  track  data  to  C2BMC  to 
support  engagement  of  a  medium-range  ballistic  missile  by 
THAAD. 

•  In  the  BMDS-level  ground  tests,  the  MDA  demonstrated 
AN/TPY-2  (FBM)  radar  capability  to  provide  real-time 
track  data  that  supported  BMDS  situational  awareness, 
BMDS  sensor  tasking,  and  GMD  engagement  planning. 
However,  a  quantitative  assessment  of  the  AN/TPY-2 
(FBM)  radar  performance  was  not  possible  since  the 
MDA  used  some  models  and  simulations  (including  the 
intermediate-range  and  intercontinental  ballistic  missile 
threat  models)  that  were  not  accredited  for  performance 
assessment. 

Cobra  Dane  Radar 

•  In  GTX-04e,  the  MDA  demonstrated  interoperability  of 
the  Cobra  Dane  radar  with  a  new  version  of  the  GMD  Fire 
Control  software.  In  addition,  the  MDA  demonstrated  a 
capability  of  the  Cobra  Dane  radar  to  provide  real-time 
data  that  enabled  the  GMD  system  to  generate  missile 
engagement  plans  and  supported  GMD  system  engagement 
of  intermediate-range  and  intercontinental  ballistic  missile 
threats.  A  quantitative  assessment  of  the  Cobra  Dane  radar 
performance  was  not  possible  since  the  MDA  used  some 
models  and  simulations  (including  the  intercontinental 
ballistic  missile  threat  models)  that  were  not  accredited  for 
performance  assessment. 

•  Due  to  its  location  and  field  of  view,  the  Cobra  Dane  radar 
has  not  participated  in  BMDS  intercept  flight  tests.  The 
MDA  plans  to  conduct  a  target  flight  test  through  the  Cobra 
Dane  radar  field  of  view  in  3QFY15  to  support  model  and 
simulation  accreditation. 

SBIRS/DSP  System 

•  SBIRS/DSP  system  performance  and  its  capability  to 
support  BMDS  engagement  of  intermediate-range  and 
intercontinental  ballistic  missile  threats  will  be  provided  in 
the  classified  appendix  of  DOT&E’s  “2012  Assessment  of 
the  Ballistic  Missile  Defense  System  (BMDS)”  report  to 
Congress. 

SBX  Radar 

•  In  GTX-04e,  the  MDA  demonstrated  interoperability  of  the 
SBX  radar  with  a  new  version  of  the  GMD  Fire  Control 
software. 

•  The  MDA  has  demonstrated  a  capability  of  the  SBX 
radar  to  support  GMD  engagement  planning  against  an 
intermediate-range  ballistic  missile  target.  However,  the 
MDA  has  not  gathered  adequate  SBX  radar  performance 
data  against  intermediate-range  and  intercontinental 
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ballistic  missile  threats  and  targets  to  enable  accreditation  of 
the  SBX  radar  models  and  simulations  that  are  required  for 
performance  assessment. 

UEWR/EWR  -  Beale,  Thule,  Fylingdales,  Clear,  and  Cape  Cod 

•  Due  to  their  locations  and  fields  of  view,  the  UEWRs  at 
Beale,  Thule,  and  Fylingdales  have  not  participated  in 
BMDS  intercept  flight  tests  in  an  operationally  realistic 
manner.  Data  from  targets  of  opportunity  and  ground  tests 
support  performance  estimates  for  the  current  configuration 
of  the  UEWRs.  These  estimates  rely  on  models  and 
simulations  that  have  not  been  fully  accredited  for  use  in 
performance  assessment. 

•  The  MDA  and  the  Air  Force  have  not  yet  upgraded  the 
EWRs  at  Clear  and  Cape  Cod  Air  Force  Stations,  and  these 
radars  are  not  yet  part  of  the  MDA’s  sensor  network. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  has 
satisfactorily  addressed  all  but  two  of  the  previous  sensor 
recommendations.  Although  the  MDA  and  Combatant 
Commanders  have  made  progress  on  developing  concepts 
of  operations  for  the  sensors,  this  FY09  recommendation 


remains  open  pending  completion  of  those  concepts  and 
implementation  in  operational  testing.  Additionally,  while 
the  MDA  has  specified  the  Aegis  BMD  LRS&T  radar  as 
the  primary  data  source  for  GMD  engagement  planning 
in  an  intercept  flight  test  attempt  in  3QFY14,  this  FY11 
recommendation  remains  open  pending  successful  execution 
of  that  test. 

•  FY12  Recommendations.  The  DOT&E  February  2012 
THAAD  and  AN/TPY-2  Radar  Operational  and  LFT&E 
Report  made  three  recommendations  for  the  MDA  and 
Army  to  consider  for  AN/TPY-2(FBM).  One  of  the 
recommendations  aligns  directly  with  the  Army  materiel 
release  conditions,  which  are  being  addressed  through  a  plan 
agreed  upon  by  the  Sensors  Product  Office  and  the  Army.  The 
following  two  recommendations  from  the  report  remain  open. 

1 .  Perform  additional  flight  testing  with  multiple  AN/TPY-2 
(FBM)  radars  in  a  single  Area  of  Regard  or  theater  to  assess 
C2BMC’s  ability  to  correctly  task  and  coordinate  track  data 
from  multiple  radars. 

2.  Conduct  independent,  dedicated  AN/TPY-2  (FBM) 
Information  Assurance  testing. 
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Live  Fire  Test  and  Evaluation  (LFT&E)  Program 


DOT&E  executed  oversight  of  survivability  and  lethality  test 
and  evaluation  for  122  acquisition  programs  in  FY12.  Of 
those  122  programs,  19  operated  under  the  waiver  provision  of 
U.S.  Code,  Title  10,  Section  2366,  by  executing  an  approved 
alternate  LFT&E  strategy  in  lieu  of  full-up  system-level  testing. 
In  addition.  Section  2366  also  requires  DOT&E  to  report  on  a 
program’s  LFT&E  results  prior  to  that  program  entering  into 
full-rate  production. 

DOT&E  published  reports  on  the  following  programs  during 
the  past  year  (reports  marked  with  an  asterisk  were  sent  to 
Congress): 

LFT&E  Reports 

•  Enhanced  Combat  Flelmet* 

•  Family  of  Medium  Tactical  Vehicles  (FMTV)  A1P2* 

•  Stryker  Double-V  Hull  (DVH)  Configuration  of  the 
Commander’s  Vehicle  (CVV) 

Special  Reports  Regarding  LFT&E 

•  Active  Protection  Systems  (APS)* 

•  Hellfire  Romeo  Missile 

•  Mine  Resistant  Ambush  Protected  (MRAP)  -  All-Terrain 
Vehicle  (M-ATV)  Underbody  Improvement  Kit  (UIK) 

•  Stryker  Mobile  Gun  System  (MGS)  Engineering  Change 
Order  (ECO)  Block  3 

•  BAE-Tactical  Vehicle  System  (TVS)  Caiman  Mine  Resistant 
Ambush  Protected  (MRAP)  Vehicle. 

Combined  OT&E/LFT&E  Beyond  Low-Rate  Initial 
Production  Reports 

•  AH-64D  Apache  Block  III  (AB3)  Attack  Helicopter* 

•  Mine  Resistant  Ambush  Protected  (MRAP)  Family  of 
Vehicles:  Dash  with  Independent  Suspension  System  (ISS), 
MRAP  Recovery  Vehicle  (MRV),  Marine  Corps  Cougar 
Ambulance* 


•  Spider  XM7  Network  Command  Munition* 

•  Stryker  Nuclear,  Biological,  and  Chemical  Reconnaissance 
Vehicle  (NBCRV)* 

•  Terminal  High- Altitude  Area  Defense  (THAAD)  and 
AN /TPY- 12  Radar* 

Combined  OT&E/LFT&E  Assessments 

•  Family  of  Medium  Tactical  Vehicles  (FMTV) 

•  MH-60S  Airborne  Mine  Countermeasures  Helicopter  and 
AN/ASQ-20AMine  Detecting  Sonar 

•  Mine  Resistant  Ambush  Protected  (MRAP)  Family  of 
Vehicles:  Navistar  Dash  Ambulance  and  MRAP  All-Terrain 
Vehicle  (M-ATV)  Underbody  Improvement  Kit  (UIK) 

•  Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Engineer 
Squad  Vehicle  (ESVV) 

•  Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Scout 
Vehicle  (ICVV) 

•  Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Mortar 
Carrier  Vehicle  (MCVV) 

•  Stryker  Double-V  Hull  (DVH)  Configuration  of  the  Medical 
Evacuation  Vehicle  (MEW) 

In  addition  to  satisfying  acquisition  oversight  requirements, 
the  LFT&E  program  funds  and  executes  technical  oversight  on 
investment  programs  that  provide  joint  munitions  effectiveness 
data  (Joint  Technical  Coordinating  Group  for  Munitions 
Effectiveness).  The  LFT&E  program  also  develops  advanced 
technologies  and  analytical  methods  to  increase  aircraft 
survivability  (Joint  Aircraft  Survivability  Program),  and  conducts 
vulnerability  and  lethality  testing  of  fielded  platforms  and 
weapons  systems  and  improves  survivability  analysis  tools  (Joint 
Live  Fire  Program).  LFT&E  investment  programs  also  support 
quick  reaction  efforts  addressing  urgent  operational  commander’s 
needs. 


JOINTTECHNICAL  COORDINATING  GROUP  FOR  MUNITIONS  EFFECTIVENESS  (JTCG/ME) 


The  Joint  Logistics  Commanders  chartered  the  Joint  Technical 
Coordinating  Group  for  Munitions  Effectiveness  (JTCG/ME) 
in  1968  to  ensure  development  of  consistent  and  credible 
effectiveness  estimates  for  conventional  munitions  across  the 
DoD.  DOT&E  oversees  the  JTCG/ME  and  provides  funding. 
The  JTCG/ME  produces  and  distributes  these  data  in  Joint 
Munitions  Effectiveness  Manuals  (JMEMs).  The  primary 
application  supported  is  weaponeering,  the  detailed  technical 
planning  of  a  weapon  strike  that  occurs  at  multiple  levels  in  the 
operational  chain  of  command  before  actual  combat.  JMEMs 
provide  computerized  tools  and  data  for  rapid  evaluation  of 
alternative  weapons  and  their  delivery  against  specific  targets. 
In  many  cases,  JMEMs  generate  collateral  damage  estimates  as 
a  part  of  the  decision  criteria  for  strikes  approved  at  the  highest 
levels  of  the  U.S.  Government. 


In  FY12,  the  JTCG/ME  published  two  updated  JMEMs  and  a  set 
of  Collateral  Effects  Radii  (CER)  tables.  The  JMEM  Weaponeering 
System  (JWS)  v2.1  software  and  the  JTCG/ME-generated 
Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3160.01  CER 
tables  are  used  for  operational  weaponeering  and  collateral 
damage  estimates  directly  supporting  operations  in  the  U.S.  Africa 
Command  and  U.S.  Central  Command  Areas  of  Responsibility.  To 
provide  continued  support  to  operational  commanders,  as  well  as 
DoD  targeting  and  mission  planners,  the  JTCG/ME  also  developed 
and  incorporated  geometric  models  representative  of  various  targets 
for  use  in  assessing  the  effectiveness  of  weapons.  JTCG/ME 
released  the  second  updated  JMEM,  JTCG/ME’s  air-to-air 
and  surface-to-air  planning  model,  the  Joint  Anti-Air  Combat 
Effectiveness  System  (J-ACE)  v5.1,  in  July  2012,  which  includes 
the  data  required  to  assess  the  survivability  of  strike  aircraft. 
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Operational  Weaponeering 

JWS  is  the  joint  source  for  air-to-surface 
and  surface-to-surface  weaponeering, 
munitions,  and  target  information. 

JWS  evaluates  the  effectiveness  of  a 
multitude  of  munition-target  combinations 
for  numerous  air-to-surface  and 
surface-to-surface  munitions  against 
a  variety  of  target  types  in  real-time. 

JWS  v2.1  contains  the  Fast  Integrated 
Structural  Tool  (FIST).  FIST  is  the 
JMEM  operational-level  tool  that  allows 
weaponeers  to  quickly  develop  geometric 
models  of  soft  and  hardened  targets  and 
use  those  models  to  evaluate  weapons 
effects.  In  particular,  the  tool  can  then 
generate  weapon  effectiveness  and  damage  assessments  for 
infrastructure  targets  including  buildings,  bunkers,  and  tunnels. 
JWS  v2.1  contains  approximately  180  new  or  updated  targets, 

15  new  and  updated  munitions,  new  Explosive  Equivalent 
Weights  based  on  blast  testing,  and  an  improved  3-D  viewer  for 
displaying  target  models. 

Operational  Mission  Planning 

JTCG/ME  released  J-ACE  v5.1  to  support  operational 
mission  planning,  particularly  at  U.S.  Strategic  Command 
(USSTRATCOM).  J-ACE  simulates  U.S.  and  threat  air-to-air 
and  surface-to-air  engagements,  providing  flyout  models  for 
U.S.  and  threat  air-to-air  and  surface-to-air  missiles,  as  well 
as  probability  of  kill  estimates.  Previous  releases  generated 
pre-computed  probability  of  kill  tables  given  an  intercept  for 
selected  weapon-target  pairings  and  engagement  conditions. 
Because  use  of  these  tables  proved  tedious,  J-ACE  v5.1  now 


provides  new  Endgame  Manager  v2.2.0 
software  and  data  sets,  which  allow 
on-demand  calculation  of  multiple  kill 
levels  for  specific  engagement  conditions 
encountered  at  intercept.  To  more 
effectively  support  operational  mission 
planning,  J-ACE  v5.1  also  provides  a 
direct  interface  to  force-level  simulations. 

These  simulations  are  used  to  study 
tactics,  evaluate  training,  assess  the 
relative  performance  of  missiles,  and  plan 
scenarios. 

Collateral  Damage  Estimation 

The  JTCG/ME  supported  development 
and  fielding  of  the  Digital  Precision  Strike 
Suite  Collateral  Damage  Estimation  tool 
for  operational  use.  This  tool  displays  accredited  Collateral 
Damage  Estimate  Level  effective  radii  reference  tables. 
Additionally,  JTCG/ME  personnel  trained  nearly  250  users  at 
10  different  commands  to  support  collateral  damage  estimation 
decisions. 

Information  Operations 

In  conjunction  with  the  Air  Force  Targeting  Center,  the 
JTCG/ME  assessed  fielded  and  emerging  applications  for 
conducting  information  operations  as  part  of  early  efforts  to 
develop  a  capability  within  JMEM  to  evaluate  the  effects  of 
such  operations.  These  efforts  included  the  development  of  an 
initial  set  of  simple  tools  for  assessing  the  effects  of  selected 
applications  when  used  to  conduct  Computer  Network  Attack, 
Computer  Network  Defense,  Military  Information  Support 
Operations,  and  Electronic  Warfare. 


JOINT  AIRCRAFT  SURVIVABILITY  PROGRAM  (J  ASP) 


DOT&E  sponsors  and  funds  the  Joint  Aircraft  Survivability 
Program  (JASP),  which  develops  techniques  and  technology  to 
improve  the  survivability  of  U.S.  military  aircraft.  The  Naval 
Air  Systems  Command,  Army  Aviation  and  Missile  Command, 
and  Air  Force  Aeronautical  Systems  Center  charter  the  program. 
DOT&E  establishes  objectives  and  priorities  for  the  JASP  and 
exercises  oversight  of  the  program.  Working  with  joint  and 
Service  staffs,  other  government  agencies,  and  industry,  the  JASP 
promotes  the  adoption  throughout  the  Military  Services  of  the 
most  current  techniques  and  technologies  for  improving  aircraft 
survivability. 

The  JASP  is  supporting  the  Joint  Multi-Role  (JMR)  Technology 
Capabilities  Demonstration  (TCD)  as  a  member  of  the  Platform 
Integrated  Product  Team.  The  purpose  of  the  JMR  TCD  is 
to  demonstrate  transformational  vertical  lift  technologies  to 
prepare  the  DoD  to  develop  the  next  generation  of  vertical  lift 
aircraft.  JASP  was  instrumental  in  establishing  assumptions  and 
requirements  for  the  vulnerability  analysis  used  in  evaluating  the 
initial  three  government  model  prototypes. 


The  JASP  funded  59  development  projects  ($10.2  Million)  and 
delivered  41  final  reports  in  FY12.  The  following  examples 
illustrate  current  JASP  efforts  in  four  focus  areas:  susceptibility 
reduction,  vulnerability  reduction,  survivability  assessment,  and 
combat  damage  assessment. 

Susceptibility  Reduction 

These  efforts  addressed  urgent  aircraft  survivability  needs 
emerging  from  Operation  Enduring  Freedom,  as  well  as 
improved  aircraft  survivability  against  future  threats. 

Countermeasure  Requirements  against  New  Infrared  Seekers. 

The  JASP,  in  conjunction 
with  the  Naval  Research 
Laboratory  (NRL),  is 
performing  in-depth  analyses 
of  newly  obtained  threat 
infrared  seekers  to  develop 
flare  and  jammer  infrared 
countermeasures  (IRCM).  The  first  objective  is  to  identify 
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anomalous  behavior  seen  when  certain  countermeasures  are  used 
against  one  particular  threat.  The  second  objective  is  to  evaluate 
the  effectiveness  of  current  countermeasure  technologies,  tactics, 
and  techniques  when  employed  against  these  seekers.  The  third 
objective  is  to  develop  new  tactics  and  techniques  to  improve 
countermeasure  performance  if  the  current  methods  are  not 
sufficient  in  defeating  the  seekers.  Initial  results  focused  on 
developing  new  countermeasure  solutions  for  one  threat  system. 

Advanced  Techniques  for  Radio  Frequency  Countermeasures 
(RFCM). 

In  partnership  with  the  U.S. 

Air  Force  Special  Operations 
Command,  the  JASP  is 
developing  and  testing  RFCM 
technology  and  techniques  to 
increase  aircraft  survivability  and 
situational  awareness  for  Army, 

Navy,  and  Air  Force  special 
operations  rotary-  and  fixed-wing 
aircraft.  This  project  assesses  the  ability  of  onboard  systems  to 
receive,  process,  and  display  each  operating  mode  of  the  threat 
weapon  system;  develops  RFCM  techniques;  and  demonstrates 
their  effectiveness  against  state-of-the-art  threat  radar  weapons. 
Validated  threat  identification  and  countermeasure  techniques 
for  surface-to-air  missile  and  airborne  radar  systems  are  being 
developed  for  incorporation  in  the  AN/ALQ-211  Suite  of 
Integrated  Radio  Frequency  Countermeasures  on  the  CV-22, 
MF1-47,  and  MF1-60,  and  in  the  AN/ALR-69  radar  warning 
receivers  on  the  C-130. 

In  partnership  with  NRL, 
the  JASP  is  demonstrating 
the  use  of  a  Digital  Radio 
Frequency  Memory  jammer 
that  adapts  in  real-time 
to  the  received  radar 
waveform,  rather  than 
identifying  the  threat  /  radar 
mode  and  selecting 
electronic  attack  techniques  from  a  pre-programmed  database. 
The  intent  is  to  overcome  limitations  of  pre-programmed 
response-based  electronic  attack  techniques  against  advanced, 
multi-mode,  coherent  radar  threats.  If  successful,  this  technique 
could  be  implemented  on  EA-18G  Growler  aircraft  as  well  as 
other  electronic  attack  platforms. 

ShotSense  3-D  Aircraft  Hostile  Fire 
Indication  (HFI)  System. 

The  JASP  and  the  U.S.  Army 
Communication-Electronics 
Research  Development  and 
Engineering  Center  supported 
development  of  a  high  performance; 
low  cost,  size,  weight,  and  power; 
uncooled  infrared  threat  detection 
system  for  the  tracking  and 


classification  of  small  arms,  rocket-propelled  grenades  (RPGs), 
missiles,  and  other  hostile  fires.  The  initial  focus  was  on  ground 
testing  in  natural  and  urban  high-clutter  environments.  Tests 
demonstrated  the  ability  to  detect  and  classify  threats  and  cue 
radars  for  projectile  tracking  during  a  Live  Fire  Demonstration. 
The  DoD  and  the  United  Kingdom  Ministry  of  Defence  are 
considering  using  the  system  for  HFI/Counter- Rocket,  Artillery 
and  Mortar  applications  at  forward  operating  bases.  The  DoD 
is  also  considering  the  system  for  use  on  the  U.S.  Special 
Operations  Command  Little  Bird  helicopter  to  provide  that  small 
aircraft  with  hostile  fire  and  missile  warning  capabilities. 

Hostile  Fire  Indication  (HFI)  Threat  System  Geolocation. 

In  partnership  with  the 
U.S.  Army  Aviation 
Applied  Technology 
Directorate,  the  JASP  is 
developing  and  testing 
software  algorithms 
for  incorporation  in 
the  AN/AVR-2B(V) 

Laser  Warning  System 
enabling  the  system  to  perform  precise  geo-location  of  hostile 
fire  threats.  The  algorithms  will  be  applicable  to  all  aircraft 
using  the  AN/AVR-2B(V),  such  as  the  Apache  and  Black  Hawk 
helicopters. 

Vulnerability  Reduction 

Several  agencies  undertook  efforts  to  develop  lighter-weight 
opaque  and  transparent  ballistic  protection  systems,  fuel 
containment  and  related  fire  protection  technologies,  and  tolerant 
structures  and  materials,  including  self-healing  composites. 

Critical  Component  Protection. 

The  U.S.  Army  Aviation 
Applied  Technology  Directorate 
led  a  project  to  manufacture 
complex,  curved  ceramic 
armor  for  placement  at 
strategic  locations  on  aircraft, 
improving  survivability 
with  minimal  weight  impact. 
These  installations  protect 
flight-critical  aircraft  components 
that  when  damaged  would 
lead  to  catastrophic  aircraft 
loss.  Due  to  their  complexity, 
these  structurally  integrated 
panels  required  development 
of  several  cutting-edge  material  and  processing  technologies. 

Two  implementations  were  demonstrated:  the  OH-58D  Kiowa 
Warrior  engine  bay  door  and  the  AH-1Z  Cobra  helicopter  flight 
control  linkage  bell-crank.  The  exposed  bell-crank  is  a  point 
of  vulnerability  in  the  flight  control  system.  The  complex 
(multiple  curvatures  in  multiple  directions)  panel  provides 
ballistic  protection.  Both  implementations  successfully  increased 
protection  with  little  or  no  weight  increase  and  are  being 
evaluated  for  transition  to  the  fleet. 


Omniscient  Electronic  Attack. 


AH-1Z  Flight  Control  Bell  Crank 


AH-1Z  Complex  Shape  Armor  Installation 
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Self-Healing  Fuel  Cell  Membrane. 

The  Naval 
Air  Warfare 
Center  Weapons 
Division  led 
development  of  a 
new  self-sealing 
coating 
technology. 

The  coating 
is  designed  to 
mitigate  leaks 

from  composite  structures  containing  fluid  (e.g.,  wet  wings  or 
fuel  compartments)  when  subjected  to  a  ballistic  impact.  The 
goal  is  to  seal  the  fuel  cell  to  a  damp  seal  within  two  minutes  at 
ambient  temperature  at  a  reduced  weight  compared  to  current 
self-sealing  fuel  bladders.  The  solution  must  be  suitable  for 
retrofit  on  fielded  aircraft  or  installation  on  new  aircraft  and 
must  work  equally  well  with  new  alternative  fuels.  Following 
successful  initial  testing,  the  Air  Force  selected  this  solution  as 
a  component  of  the  vulnerability  reduction  design  for  the  future 
KC-46  tanker  aircraft. 

Thermal  Degradation  of  Composites. 

This  project  examined 
the  degradation  of  aircraft 
structural  composite  materials 
due  to  heat  damage  caused  by 
short-lived  fuel  fires.  After 
calibration  through  mechanical 
test,  evaluations  were  made 
of  various  non-destructive 
inspection  (ND1)  methods  and 
their  ability  to  detect  incipient  thermal  degradation.  The  results 
of  the  testing  were  transitioned  to  Fleet  Readiness  Centers  for 
the  generation  of  inspection  and  repair  procedures.  The  success 
of  this  project  also  led  to  a  U.S. -Germany  bilateral  research 
agreement  and  a  new  project  to  validate  fire  and  heat  exposure 
damage  data  on  representative  composites  used  extensively 
on  the  F-35.  The  new  effort  will  also  validate  the  ability  of 
portable  NDI  techniques  to  detect  equivalent  measures  of  thermal 
degradation. 

Survivability  Assessment 

The  projects  in  this  area  develop  and  maintain  components 
of  models  and  simulations  that  are  widely  used  to  support 
weapon  system  acquisition  (e.g.,  design  studies,  specification 
development,  and  specification  compliance)  and  test  and 
evaluation  (e.g.,  test  design,  evaluation,  and  assessment). 

Suite  of  Anti-Air  Kill  Chain  -  Models  and  Data  (SAK-MD). 

The  JASP,  the  JTCG/ME,  and  USSTRATCOM  have  been 
working  for  several  years  to  improve  the  methods  and  data 
employed  by  USSTRATCOM  to  plan  global  strike  missions. 
Initial  efforts  developed  a  new  methodology  to  efficiently 
and  consistently  assess  probability  of  kill  for  a  large  set  of 
blue  aircraft  and  red  threat  pairs.  More  recently,  the  team 
has  concentrated  on  adding  the  effects  of  radio  frequency 


and  infrared  countermeasures  into  the  methodology  and  data 
sets.  By  coupling  widely  accepted  countermeasure-capable 
engagement  models  with  fast  running  operational  user-focused 
models,  the  team  has  developed  a  new  benchmark  capability  for 
performing  aircraft-threat  engagement  analyses.  SAK-MD  has 
been  widely  distributed  as  part  of  the  JTCG/ME  J-ACE  product 
and,  in  addition  to  USSTRATCOM,  is  used  extensively  by  Air 
Force,  Navy,  and  Marine  Corps  aircrews  in  weapon  systems 
training. 

Crew  and  Passenger  Survivability  (CAPS). 

The  JASP  continued  to  advance  the  development  of  analysis, 
data,  test  capability,  and  technology  for  improving  aircraft 
CAPS.  This  year  concluded  the  initial  effort  to  develop  an 
analytical  framework  for  CAPS  evaluation  by  demonstrating  the 
inclusion  of  CAPS  in  two  commonly  used  vulnerability  analysis 
models.  The  effort  also  included  drafting  a  report  on  the  state 
of  crew  casualty  evaluation  methodologies  and  a  roadmap  for 
future  methodology  development.  The  project  will  continue 
to  improve  CAPS  assessment  methodologies,  test  capabilities, 
and  technologies  working  in  partnership  with  the  National 
Aeronautics  and  Space  Administration,  Federal  Aviation 
Administration,  and  U.S.  Army  Tank  Automotive  Research, 
Development  and  Engineering  Center. 

Infrared  Countermeasure  (IRCM)  Modeling. 

The  JASP 
continued 
data  collection 
and  model 
development 
to  improve 
the  DoD’s 
IRCM  analysis 
capability 
against 
advanced 

multi-spectral  and  imaging  infrared  threats.  Current  infrared 
engagement  simulations  rely  on  simple  flare  models  that  lack 
the  resolution  required  to  address  advanced  infrared  missile 
seekers.  The  JASP  and  Naval  Surface  Warfare  Center  -  Crane 
Division  are  developing  physics-based  models  for  pyrotechnic 
and  pyrophoric  flares  that  address  combustion,  heat  and  mass 
transfer,  infrared  radiation,  trajectory,  and  spatial  extent/image 
presentation;  ultimately  providing  the  time  evolution  of  plume  or 
cloud  characteristics  necessary  for  analysis  of  the  effectiveness 
of  countermeasures  against  missiles  using  advanced  infrared 
seekers.  The  project  is  also  modernizing  the  joint  modeling  and 
simulation  tools,  Flare  Aerodynamic  Modeling  Environment, 
and  the  Tri-Service  Flare  Database  to  improve  usability,  add  3-D 
descriptions  of  air  flowing  around  aircraft,  and  develop  Linux 
versions  of  both  tools. 

Combat  Damage  Assessment 

JASP  continued  to  support  the  Joint  Combat  Assessment  Team 
(JCAT)  in  FY12.  JCAT  comprises  Army,  Air  Force,  Navy, 
and  Marine  Corps  personnel  deployed  in  support  of  combat 
operations.  JCAT  continued  its  operation  in  Afghanistan 


V-22  Dispensing  Flares  Clouds 
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with  full-time  deployments  in  Regional  Commands  -  South, 
Southwest,  and  East.  Iraq  and  other  areas  of  the  world  were 
supported  remotely  or  by  rapid  deployment  from  Afghanistan  or 
stateside. 

JCAT  inspects  damaged  and  destroyed  aircraft,  acquires 
maintenance  records,  and  conducts  interviews  with  aircrew  and 
intelligence  personnel  to  develop  an  accurate  and  comprehensive 
assessment  of  each  aircraft  combat  damage  event.  They  provide 
consultation  to  weapons,  tactics,  and  logistics  personnel  and 
comprehensive  briefings  to  commanders  in  charge  of  daily  air 
operations.  These  efforts  allow  battlefield  commanders  to  adjust 
operational  tactics,  techniques,  and  procedures  based  on  accurate 
threat  and  damage  assessments.  As  of  November  22,  2012,  the 
JCAT  had  initiated  221  and  completed  179  FY12  aircraft  combat 
damage  assessments. 

The  JCAT  improved  aircraft  combat  damage  incident  reporting 
in  the  Services  and  the  DoD.  The  Combat  Damage  Incident 


Reporting  System  (CD1RS)  hosted  by  the  Survivability/ 
Vulnerability  Information  Analysis  Center  (SURVIAC)  is  the 
repository  for  all  U.S.  aircraft  combat  damage  reports.  The  JCAT 
worked  closely  with  SURVIAC  to  upgrade  the  CDIRS  database, 
its  data  reduction  capability,  and  links  to  the  JCAT  knowledge 
centers.  JCAT  and  SURVIAC  are  also  working  with  OSD  and 
U.S.  Central  Command  (USCENTCOM)  on  an  operational 
demonstration  linking  both  CDIRS  and  USCENTCOM  databases 
to  more  quickly  identify,  assess,  document,  and  distribute  aircraft 
combat  damage  incident  data  to  the  Services  and  DoD. 

The  JCAT  trains  the  U.S.  aviation  community  on  potential  aircraft 
threats  and  combat  damage,  including  representatives  from  the 
U.S.  Military  Services,  the  Department  of  State,  the  Department 
of  Homeland  Security,  the  Federal  Aviation  Administration,  the 
Department  of  Energy,  the  Federal  Bureau  of  Investigation,  and 
the  Bureau  of  Alcohol,  Tobacco,  Firearms  and  Explosives. 


JOINT  LIVE  FIRE  (JLF) 


The  goal  of  the  Joint  Live  Fire  (JLF)  program  is  to  test 
fielded  systems,  identify  vulnerable  areas,  understand  damage 
mechanisms,  and  provide  the  information  needed  to  make  design 
changes;  modify  tactics,  techniques,  and  procedures;  or  improve 
analytical  tools.  The  need  for  these  tests  result  from  systems 
being  exposed  to  new  threats,  being  used  in  new,  unanticipated 
ways,  or  being  operated  in  new  combat  environments,  thereby 
requiring  an  updated  assessment  of  their  performance. 

JLF  supplements  LFT&E  of  systems  by  testing  new  threats  that 
the  requirements  community  did  not  anticipate  during  original 
development  or  old  threats  employed  in  new  ways.  The  RPG 
is  an  example  of  a  threat  employed  differently  than  initially 
intended.  Originally  developed  as  an  anti-tank  or  anti-personnel 
weapon,  hostile  forces  in  Afghanistan  often  use  the  RPG  as  an 
anti-helicopter  weapon. 

Aircraft  Systems  Program 

JLF-Air  completed  nine  test  series  in  FY12,  with  a  focus  on 
resolving  key  modeling  and  testing  deficiencies  highlighted 
in  the  2010  Man-Portable  Air  Defense  Systems  (MANPADS) 
Vulnerability  Capabilities  Roadmap,  providing  threat 
characterization  data  for  high-explosive  incendiary  and  armor 
piercing  projectiles,  and  investigating  occupant  survivability 
deficiencies  resulting  from  ballistic  events.  The  JLF  program 
has  made  significant  progress  over  the  last  three  years  in 
understanding  MANPADS  threat  characteristics  and  developing  a 
viable  test  capability  for  use  in  LFT&E. 

Large  Engine  Vulnerability  to  MANPADS. 

This  project  is  assessing  the  ballistic  vulnerability  of  a  large 
turbofan  engine  to  a  MANPADS  missile  impacting  from  the  rear, 
and  using  that  information  to  explore  subsequent  flying  qualities 
of  large  aircraft.  Live  fire  testing  using  realistic  engagement 
conditions  were  achieved  by  using  shotlines  representative  of 


real-world  encounters, 
operating  the  engine 
at  typical  thrust  levels, 
simulating  the  proper 
amount  of  external 
airflow  across  and 
through  the  engine,  and 
accurately  controlling 
the  MANPADS  missile 
impact  location,  velocity, 
and  detonation  point. 

The  project  provided  test  results  to  the  National  Aeronautics 
Space  Administration  enabling  it  to  conduct  flight  simulations 
to  determine  the  flying  qualities  that  can  be  expected  during 
cruise  and  landing  as  a  result  of  the  damage  witnessed  in  the 
test.  The  project  will  present  simulation  results,  along  with  test 
observations,  in  a  final  report  to  be  completed  in  early  2013. 

MANPADS  Threat  Model  Development  -  Fragment  and  Debris. 
This  project  concluded  testing  for  the  MANPADS  Threat  Model 
Development  test  series.  This  test  series  was  designed  to 
gather  data  with  sufficient  quality  to  improve  the  accuracy  and 
credibility  of  MANPADS  threat  models  used  to  assess  and  predict 
aircraft  vulnerability.  This  test  collected  warhead  fragment  and 
debris  for  two  MANPADS  that  were  flown  and  detonated  in  the 
center  of  a  specially  instrumented  test  arena. 

Supersonic  Rocket  on  a  Rope. 

This  project  is  developing  a  capability  to  support  testing 
of  various  platforms’  vulnerability  to  MANPADS  missiles. 

Using  this  technique,  the  missile  is  flown  along  a  set  of  ropes, 
acting  as  guide  wires,  which  are  cut  just  before  reaching  the 
intended  target.  Once  cut  free,  the  missile  is  allowed  to  free-fly 
into  the  target  achieving  a  high-level  of  precision  in  hitting 
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pre-determined  impact  points.  Prior  year  tests  satisfactorily 
demonstrated  high  precision  hits.  This  year’s  tests  also  included 
detonating  the  MANPADS  missile  warhead  using  a  range-safe 
ftize.  In  2013,  the  project  will  be  in  its  final  year  and  is  expected 
to  culminate  in  a  demonstration  with  three  shots  using  actual 
MANPADS  tactical  fuzes,  rather  than  range  fuzes,  for  warhead 
detonation. 

Threat  Projectile  Characterization. 

Several  JLF-Air  projects  focused  on  collecting  threat 
characterization  data  for  small  arms  or  Anti-Aircraft  Artillery 
(AAA)  threats.  One  such  project  filled  deficiencies  in  data  on 
the  performance  of  certain  fuzes  used  in  14.5  mm  and  23  mm 
high-explosive  incendiary  projectiles.  A  gap  in  understanding 
the  ballistic  performance  multiple  variations  of  7.62  mm  and 
.30-caliber  armor  piercing/armor  piercing  incendiary  projectiles 
similarly  resulted  in  a  test  series  to  determine  the  appropriate 
round  for  use  in  experimental  ballistic  testing. 

V50  Yaw  of  Projectiles. 

This  project  is  exploring 
how  a  projectile,  such 
as  those  impacting 
fast  moving  targets, 
penetrates  typical 
aircraft  materials  when 
impacting  the  materials 
at  other  than  normal  (i.e., 

90  degrees)  incidence. 

Test  results  will  provide 
immediate  feedback 
on  the  accuracy  of  the  analytical  vulnerability  tools  now  being 
commonly  used  and  will  be  used  to  update  the  Computation 
of  Vulnerable  Area  Tool  and  the  Advanced  Joint  Effectiveness 
Model.  The  test  concept  was  successfully  demonstrated  in  2012 
and  40  test  events  at  varying  speeds,  effectively  creating  different 
yaw  angles,  are  underway. 

Rotorcraft  Sponson  RPG  Vulnerability. 

This  project  evaluated  three  technologies  for  suppressing  cabin 
fires  resulting  from  RPG  impacts  to  sponson  fuel  tanks  adjacent 
to  aircraft  cabins.  Although  large  main  cabin  fires  occurred 
in  this  testing,  the  testing  demonstrated  that  some  of  these 
technologies  are  effective  in  either  mitigating  or  eliminating 
the  resulting  fire.  This  testing  focused  on  cabin  environment 
and  occupant  survivability.  Data  from  cabin  thermocouples, 
pressure  transducers,  and  mannequins  were  collected  to  establish 
occupant  effects.  These  data,  along  with  an  evaluation  of  the  fire 
suppression  technologies  tested,  will  be  published  in  FY13. 

Crew  Compartment  Fire  Survivability. 

This  project  characterized  the  environment  of  an  aircraft  main 
cabin  during  fires  representative  of  those  seen  as  a  result  of 
ballistic  events.  Baseline  testing  focused  on  cabin  conditions 
with  no  airflow  (i.e.,  all  hatches,  doors,  and  windows  closed) 
and  the  effectiveness  of  personal  fire  extinguishers  to  extinguish 
those  fires.  Using  a  specially  modified  H-3  helicopter  fuselage 


developed  under  this  project, 
data  were  collected  using  a 
comprehensive  instrumentation 
package  gathering  multiple 
temperature,  oxygen  and 
carbon  monoxide  levels,  and 
visibility  measurements.  For 
each  test  event,  the  cause  and 
time  to  reach  an  incapacitation 
threshold  were  determined. 

Ground  Systems  Programs 

Roadmap  to  Address  Hybrid-Ill  Mannequin  Sustainment. 

This  project  investigated  modification  and 
improvement  options  for  the  currently  utilized 
Hybrid-Ill  Anthropomorphic  Test  Device 
(ATD).  The  Hybrid-Ill  ATD  is  the  primary 
instrument  for  gathering  accelerative  injury 
data  from  live  fire  underbody  blast  testing. 

This  project  identified  several  low  cost 
technical  upgrades  that  will  help  sustain  the 
Hybrid-Ill  ATD  through  2018,  at  which  point 
the  live  fire  community  expects  to  transition 
to  the  Warrior  Injury  Assessment  Manikin 
(WIAMan)  ATD,  currently  under  development. 

Test  and  Analysis  to  Update  Warhead  Characterization  for 
MK  82  Bomb. 

This  project  conducted  a  series  of  five  MK  82  general  purpose 
bomb  tests  to  characterize  fragment  size,  speed,  and  direction. 
Tests  were  conducted  utilizing  the  JTCG/ME  standard 
procedures,  with  fragments  collected  in  recovery  bundles  and 
impact  speeds  measured  utilizing  electronic  triggering  panels. 

The  JTCG/ME  Systems  Characterization  Working  Group  is 
reviewing  the  results,  which,  when  accepted,  will  be  utilized 
for  weapon  lethality,  collateral  damage,  and  risk  assessments. 

This  testing  was  in  direct  response  to  requests  from  operational 
Commanders  employing  the  MK  82. 

Testing  to  Gather  Penetration  for  Low-Collateral  Damage 
Round. 

This  project  conducted  a  series  of  tests  with  low-collateral 
damage  rounds  against  plate  arrays  typical  of  a  light  ground 
mobile  target  to  obtain  residual  properties  of  the  projectiles  (i.e., 
speed,  weight,  and  orientation  after  penetration  through  different 
materials).  Information  collected  will  be  utilized  to  improve 
the  JTCG/ME  predictive  Projectile  Penetration  (ProjPen) 
model.  ProjPen  is  utilized  to  predict  penetrator  performance 
against  ground  mobile,  rotary-  and  fixed-wing  aircraft,  and  small 
watercraft.  ProjPen  currently  lacks  predictive  capability  for  this 
type  of  round  and  data  did  not  exist  to  implement  that  capability. 

Behind  Helmet  Blunt  Trauma  Skull  Injury. 

This  JLF  project  is  part  of  a  series  of  experiments  to  improve  the 
testing  and  evaluation  of  military  combat  helmets.  Blunt-ballistic 
impact  tests  will  be  used  to  develop  an  injury  criterion  or  risk  for 
behind-helmet  blunt  trauma,  where  currently  none  exist.  JLF  can 
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use  the  injury 
criterion  to 
assess  the 
severity  of 
loading, 
as  well  as 
the  risk 

of  sustaining  injury,  with  potential  to  implement  this 
criterion  into  future  testing  and  evaluation  of  new  protective 
equipment. 


Improvements  to  Sustained  Fire  Prediction  in 
Vulnerability/Lethality  Analysis  Methodologies. 

This  project  assessed  the  ability  of  lethality  and  vulnerability 
(L/V)  models  to  predict  sustained  fire  as  a  kill  mechanism 
for  ground  mobile  targets.  Analysis  of  available  test  data 
indicates  current  L/V  fire  start  methodologies  under-predict 
sustained  fire  resulting  from  near-field  detonations  of 
blast-frag  warheads.  This  initial  assessment  of  fire  start 
predictive  capability  will  compile  available  relevant  test  data 
and  reports,  assess  the  current  L/V  fire  start  models  in  light  of 
their  underlying  empirical  data,  and  propose  a  way  forward 
for  improving  the  capability  of  L/V  codes  to  predict  sustained 
fire  as  a  kill  mechanism. 


Continued  Improvement  of  Material  Characterization  for 
ProjPen  Modeling. 

This  project  is  conducting 
tests  to  gather  data  for 
small  caliber  armor  piercing 
incendiary  projectiles  striking 
stainless  steel  plates.  Testing 
is  focused  on  penetration 
velocities  and  the  gathering  of 
residual  masses  and  velocities 
of  penetrating  fragments. 

This  will  improve  the  quality 
of  L/V  analyses  involving  ProjPen.  Both  the  JTCG/ME  and 
JASP  utilize  ProjPen  for  the  analysis  of  the  damaging  effects 
of  small  caliber  projectiles. 

Effects  of  Buried  Homemade  Explosive  (HME)  Performance 
Variability  on  Platform  and  Occupant  Survivability. 

This  project  is  quantifying  the  potential  variations  in  the 
buried  blast  performance  of  materials  manufactured  to 
represent  a  typical  Homemade  Explosive  (HME).  Similar  to 
many  HMEs,  the  material  is  a  blend  of  different  materials  and 
is  granular  in  nature.  The  characteristics  being  investigated 
arise  from  the  processes  required  to  manufacture  the  material. 
Multiple  blends  at  extreme  tolerances  of  acceptable  variances 
in  mixtures  are  being  tested  in  controlled  blast  experiments. 
The  results  of  the  tests  will  be  used  to  establish  models  for 
the  performance  of  the  material.  For  effects  identified  as 
significant  (i.e.,  greater  than  standard  shot-to-shot  variability) 
the  explosive  models  will  be  used  with  a  full  system  model  to 
establish  the  effect  on  ballistic  effects  delivered  to  a  vehicle 
such  as  those  that  may  be  observed  during  a  live  fire  test. 


Sea  Systems  Program 

The  Joint  Live  Fire  Sea  Systems  Program  (JLF-Sea)  made 
significant  progress  in  FY12  towards  improving  the  capability  to 
assess  the  survivability  of  submarines  and  surface  ships.  These 
projects  benefit  ship  and  submarine  acquisition  programs  as  well 
as  the  fleet  of  fielded  U.S.  Navy  vessels. 

Finnish  Fast  Attack  Craft  Testing. 

FY12  was  the  last  year  of  a  multiyear,  trilateral  (United  States, 
Finland,  and  Germany)  cooperative  effort  to  perform  damage 
testing  against  two  aluminum,  decommissioned  Finnish  fast 
attack  craft.  The  Finnish  Navy  provided  the  ships  and  has 
conducted  testing  on  their  test  range  in  the  Baltic  Sea  and  at  their 
test  center  in  Niinisalo,  Finland.  The  objective  was  to  understand 
the  behavior  of  aluminum  structures  subjected  to  various  weapon 
effects.  In  FY12,  shaped  charge  testing  against  a  section  of 
one  of  the  ships  was  conducted  to  study  spall  and  debris  from 
aluminum  targets,  to  perform  crew  casualty  assessments,  and  to 
evaluate  depth-of-penetration.  These  tests  will  fill  gaps  in  our 
understanding  of  weapon  effects  against  aluminum  ships,  and 
will  augment  the  LFT&E  programs  for  the  Littoral  Combat  Ship 
and  Joint  High  Speed  Vessel. 

Vulnerability  of  Aluminum  Structure  to  Fire. 

This  project  conducted  tests  to 
determine  if  fire  insulation  and  its 
attachment  could  be  defeated  by  a 
major  fire.  A  test  article  representative 
of  a  section  of  a  USS  Independence 
class  Littoral  Combat  Ship  flight 
deck  was  built  using  specified 
materials  and  insulation  attachment 
materials  and  procedures.  The  article  was  then  exposed  to  tests 
similar  to  those  specified  by  military  standard  (MIL  STD  3020). 
Results  from  this  project  will  fill  gaps  in  our  understanding  of 
the  vulnerabilities  of  aluminum  structures  and  their  insulation 
systems  to  weapon-induced  fires. 

Crew  Casualty  Modeling  and  Simulation  Validation  Testing. 

The  objective  of  this  project,  jointly 
funded  by  the  Office  of  Naval 
Research  and  JLF,  is  to  validate  injury 
response  functions  developed  under  an 
earlier  JLF  project.  Three  Hybrid-Ill 
ATDs  were  tested  on  a  floating 
shock  platform  to  collect  data  on  personnel  injuries  caused  by 
an  underwater  explosion.  The  data  and  observations  from  this 
testing  will  improve  the  accuracy  of  crew  casualty  predictions 
made  as  a  part  of  ship  live  fire  vulnerability  assessments. 

Network  Fire  Model  Enhancements. 

This  project  provided  funds  to  continue  development  of  the  NRL 
network  fire  model.  The  model  can  be  used  by  naval  engineers 
to  develop  ship  designs  that  limit  the  spread  of  fire  and  smoke. 
NRL  began  a  two-year  effort  to  improve  the  model’s  capability 
to  predict  fire  characteristics  associated  with  large  volume  spaces 
such  as  hangar  and  mission  bays. 
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Sea  Bottom  Underwater  Explosion  Effects  Testing. 

This  project,  the  result  of  an  agreement  between  the  U.S.  and 
German  Navies,  continues  development  and  validation  of 
simulation  tools  for  assessing  ship  survivability  to  various 
explosive  threats.  JLF  provided  funding  to  conduct  underwater 
explosion  testing  representing  charges  located  on  and  near  the 
sea  floor  to  quantify  the  loading  on  vessels  submerged  near 
the  bottom,  submerged  between  the  surface  and  the  bottom, 


and  floating  at  keel  depth. 
This  project  provides  data 
to  increase  the  fidelity 
of  models,  increase  the 
accuracy  of  survivability 
assessments,  and  to  address 
urgent  operational  needs. 


LFT&E  SPECIAL  INTEREST  PROGRAMS 


Active  Protection  Systems  (APS) 

In  response  to  FY08  legislation,  DOT&E  tested  and  evaluated 
seven  foreign  and  domestic  (two  foreign,  three  domestic,  and  two 
combined  foreign/domestic)  APS  with  the  potential  to  protect 
wheeled  tactical  vehicles  from  RPGs.  DOT&E  provided  a  report 
to  Congress  and  the  Department’s  acquisition  leadership  in 
February  2012.  This  effort  concluded  that  none  of  the  tested  APS 
are  currently  mature  enough  for  fielding  on  any  U.S.  platform  and 
further  development,  test,  and  evaluation  are  required. 

Personnel  Protection  Equipment 

DOT&E  continued  its  oversight  of  the  testing  of  personnel 
protection  equipment.  The  Services  and  Special  Operations 
Command  (USSOCOM)  continue  to  implement  rigorous, 
statistically-based  testing  protocols  for  hard  body  armor  inserts 
and  military  combat  helmets  approved  by  DOT&E.  The  Defense 
Logistics  Agency  has  utilized  the  hard  armor  testing  protocol  in 
new  contracts  for  sustainment  stocks  of  hard  armor  inserts.  The 
Army  has  incorporated  the  key  concepts  of  statistical  confidence 
and  test  design  into  its  requirements  for  its  Soldier  Protection 
System.  The  Navy  and  Air  Force  are  also  implementing 
statistical  testing  concepts  for  soft  armor  vests. 

DOT&E,  in  partnership  with  the  Services  and  USSOCOM,  is 
developing  a  new  soft  armor  vest  testing  protocol.  That  protocol 
is  intended  to  standardize  testing  and  assure  that  soft  armor 
vests  provided  to  Service  members  meet  common  ballistic 
protection  requirements  and  provide  uniform  protection  on  the 
battlefield.  The  implementation  of  this  protocol  will  further 
increase  government  oversight  of  personal  protective  equipment 
by  requiring  soft  armor  vests  (in  addition  to  hard  armor  plates 
and  combat  helmets)  to  meet  rigorous  statistical  measures  of 
performance. 

The  National  Academy  of  Sciences’  (NAS)  Committee  to 
Review  the  Testing  of  Body  Armor  Materials  for  Use  by  the 
U.S.  Army  published  its  final  report  in  May  2012.  This  report 
completes  the  NAS’  three-phase  review  of  body  armor  testing 
that  began  in  2009  following  the  release  of  a  critical  2009  U.S. 
Government  Accountability  Office  report  of  testing  conducted 
at  the  Army’s  Aberdeen  Test  Center.  The  final  NAS  report 
reviewed  a  number  of  topics,  including  the  use  of  statistically 
principled  testing,  measurement  standards,  and  body  armor  test 
methodology.  The  report  concluded  that  the  hard  body  armor 
test  protocol  is  statistically  principled  and  provides  an  acceptable 
minimum  DoD-wide  body  armor  test  standard.  The  report  notes 


the  need  for  additional  work  in  the  areas  of  both  measurement 
improvement  and  characterizing  variance  in  the  performance 
of  the  clay  backing  material  used  during  testing  to  further 
improve  test  quality.  The  Army  is  currently  testing  a  new  clay 
formulation  that,  if  successful,  would  eliminate  the  drawbacks 
of  the  clay  currently  used  and  described  in  the  NAS  report.  The 
report  also  makes  several  recommendations  for  research  and 
investigation  into  the  medical  effects  of  blunt  force  trauma,  and 
for  applying  the  results  of  this  research  to  improve  body  armor 
test  procedures.  Both  DOT&E  and  the  Army  are  working  to 
use  the  results  of  ongoing  medical  research  into  blunt  trauma  to 
update  body  armor  test  methodologies  and  procedures.  DOT&E 
has  asked  the  NAS  to  review  and  comment  on  the  Department’s 
statistically-based  combat  helmet  test  protocol,  a  15-month  study 
that  is  expected  to  be  complete  by  the  end  of  2013. 

Stryker  Double-V  Hull  (DVH) 

The  Army  initiated  the  Stryker  Double-V  Hull  (DVH)  program  in 
response  to  Operation  Enduring  Freedom  commanders’  requests 
for  improved  underbody  blast  threat  protection.  Under  DOT&E 
oversight,  the  Army  incrementally  developed  and  tested  multiple 
Stryker  DVH  configurations  over  a  two-year  timeframe.  The 
test  and  evaluation  program  directly  supported  critical  fielding 
schedules,  and  DOT&E  reported  to  the  USD(AT&L)  and  Army 
leadership  on  each  Stryker  DVH  configuration’s  survivability, 
effectiveness,  and  suitability  prior  to  its  fielding.  Test  and 
evaluation  confirmed  that  DVH  configurations  significantly 
improve  crew  protection  against  lEDs  and  demonstrated  the 
retention  or  improvement  of  required  operational  characteristics. 
Relative  to  flat-bottom  Strykers,  Stryker  DVH  remained  equally 
mobile  and  capable  of  supporting  a  unit’s  ability  to  accomplish  a 
mission,  while  providing  improved  reliability  and  maintainability. 
The  results  of  the  test  and  evaluation  program  also  led  to 
design  changes  that  corrected  significant  shortcomings  in  DVH 
suitability  and  survivability. 

Underbody  Blast  Testing 

DOT&E  continues  to  actively  pursue  efforts  to  improve  LFT&E 
employing  underbody  blast  threats.  In  January  2011,  the  DoD 
provided  funds  to  the  Army  to  execute  a  DOT&E-sponsored 
five-year  research  and  development  program.  This  program  will 
substantially  increase  the  Department’s  understanding  of  the 
cause  and  nature  of  injuries  incurred  in  underbody  blast  combat 
events  and  will  develop  new  instrumentation  capable  of  being 
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used  to  accurately  assess  such  injuries  in  testing.  This  program, 
known  as  WIAMan  (Warrior  Injury  Assessment  Manikin), 
utilizes  expertise  across  multiple  commands  and  disciplines 
within  the  Army  to  generate  a  medical  research  plan  from  which 
data  will,  at  pre-determined  times,  be  transitioned  to  the  materiel 
and  test  and  evaluation  communities.  These  data  will  feed  the 
design  of  a  biofidelic  prototype  ATD  designed  to  accurately 
capture  occupant  loading  from  the  vertical  direction,  reflecting 
the  primary  load  axis  to  which  occupants  are  exposed  in  an 
under-vehicle  blast  event. 


Similarly,  as  a  result  of  DOT&E  efforts,  the  Army  will 
characterize  the  performance  of  a  surrogate  for  the  Afghanistan 
IED  threat  to  enable  its  use  in  test,  the  performance  of  buried 
TNT  blast  threats  (current  test  standard),  and  the  effects  of  soil 
composition  and  condition  on  full-scale  buried  test  threats.  This 
effort  will  ensure  adequate  LFT&E  in  support  of  OEF  as  well  as 
of  future  acquisition  programs. 
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Information  Assurance  (IA)  and  Interoperability  (IOP) 


In  FY12,  the  DOT&E  Information  Assurance  (IA)  and 
Interoperability  (IOP)  Assessment  Program  performed  20 
assessments  during  Combatant  Command  (CCMD)  and 
Service-level  exercises  or  real-world  activities;  3  of  these 
assessments  involved  units  deployed  to  the  U.S.  Central 
Command  (USCENTCOM)  area  of  responsibility.  The 
LA/IOP  program  conducted  reduced-scale  assessments  at  U.S. 
European  Command  (USEUCOM)  and  U.S.  Africa  Command 
(USAFRICOM)  after  their  scheduled  exercises  were  cancelled  in 
response  to  actual  operational  contingencies.  Six  individual  test 
and  evaluation  organizations  conducted  these  assessments,  which 
involved  all  10  of  the  CCMDs  and  all  4  Services.  During  the 
year,  DOT&E  released  five  major  findings  reports,  and  initiated 
another  nine,  pertaining  to  both  IA  and  IOP.  Exercise  planners 
in  FY12  made  increased  use  of  cyber  ranges  in  support  of  these 
exercises. 

Summary  of  Findings 

Most  exercise  assessments  and  tests  involved  operations  largely 
against  low-  and  mid-level  cyber  threats  and  on  networks  that 
were  only  moderately  stressed  in  terms  of  loading  or  network 
degradation;  high-level  threats  were  portrayed  infrequently.  No 
exercises  were  seriously  disrupted  by  adversary  activities,  or 
disrupted  at  any  length,  because  adversary  teams  were  generally 
not  permitted  to  take  actions  that  could  disrupt  exercises.  In  the 
cases  where  the  adversary  team  portrayed  higher-level  threats, 
exercise  training  audiences  frequently  misinterpreted  these 
portrayals  as  maintenance  issues,  poor  system  performance,  or 
anomalies. 

Overall,  the  DOT&E  LA/IOP  program  observed  cyber  effects 
caused  by  unresolved  interoperability  deficiencies,  coupled  with 
low-to-moderate  level  threats  that  were  sufficient  to  adversely 
affect  the  quality  and  security  of  mission  critical  information 
in  a  way  that  could  (and  where  permitted  did)  degrade  mission 
accomplishment  significantly.  Therefore,  considering  both 
IA  and  IOP  attributes,  the  Department  has  not  yet  developed 
sufficiently  advanced  cyber  defensive  tactics  to  counter  advanced 
adversary  tactics  and  to  consistently  operate  in  degraded  cyber 
environments. 

Interoperability:  The  FY12  IOP  assessments  documented 
interoperability  problems  involving  mission  critical  systems, 
but  these  problems  hindered  rather  than  prevented  mission 
accomplishment.  This  is  due  primarily  to  system  operators 
who  developed  workarounds  to  preserve  the  critical  mission 
functions.  Even  though  operators  accomplished  their  missions, 
the  workarounds  usually  increased  operator  workload,  and  often 
degraded  efficiency  in  completing  mission  tasks.  The  assessment 
teams  documented  effects  on  the  timeliness,  accuracy,  and 
efficiency  of  operational  data  handling.  Operators  frequently 
viewed  interoperability  problems  as  maintenance  or  design  issues 
and  therefore  did  not  report,  document,  or  remediate  many  of 


these  problems.  The  majority  of  systems  encountered  during 
assessments  were  not  certified  for  interoperability. 

Information  Assurance:  The  overall  IA  performance  observed 
during  the  FY12  exercise  assessments  remains  insufficient  to 
prevent  and  consistently  detect  compromise  and  exploitation  of 
the  networks  exercised.  Although  regularly  able  to  penetrate  and 
exploit  networks,  Red  Teams  reported  modest  increases  in  the 
required  level  of  effort  over  previous  years.  While  compliance 
with  network  standards  continues  to  improve,  the  IA/IOP  program 
continued  to  provide  low  ratings  for  certain  critical  compliance 
areas.  In  addition,  development  of  the  more  sophisticated 
tactics  and  procedures  necessary  to  counter  a  determined  or 
well-resourced  cyber  adversary  remains  slow.  In  exercises 
involving  portrayal  of  more  sophisticated  threat  profiles,  the 
training  audiences  usually  lacked  commensurate  defensive  tactics. 
Overall,  the  implementation  of  Joint  Staff  guidance  on  exercise 
realism  has  been  slow.  Network  boundary  defenses  continued  to 
improve  in  FY12,  to  include  the  presence  of  host-based  intrusion 
detection  tools,  improved  configuration  management  of  networks 
and  security  tools,  and  the  infrastructure  supporting  the  networks. 
In  at  least  one  exercise,  good  network  “housekeeping”  effectively 
deterred  adversary  efforts.  However,  DOT&E  observed  reduced 
rates  of  compliance  in  the  use  of  software  and  hardware  backups; 
and  key  practices  such  as  port-and-protocol  protections,  reliable 
software  baselines,  remediation  of  known  vulnerabilities,  and 
effective  use  of  system  audit  logs. 

Partnerships  and  Coordination 

DOT&E  continued  a  number  of  partnerships  directly  related  to  the 
conduct  of  IA/IOP  assessments.  These  included: 

•  Collaborating  with  the  Joint  Staff  and  DoD  Deputy  Chief 
Information  Officer  (CIO)  concerning  oversight  and 
coordination  of  the  I A  and  IOP  Assessment  Program.  DOT&E 
provides  metrics  and  observations  generated  from  these 
assessments  to  the  DoD  CIO  for  use  in  enterprise-wide  IA 
assessments  and  programs. 

•  Coordinating  program  efforts  with  USD(AT&L)  and 
Developmental  Test  and  Evaluation  (DT&E)  as  a  means  of 
supporting  the  acquisition  and  development  of  information 
handling  systems. 

•  Creating  a  standing  memorandum  of  understanding  between 
DOT&E  and  U.S.  Cyber  Command  (USCYBERCOM) 
that  directs  the  establishment  and  operation  of  the  Cyber 
Assessment  Synchronization  Working  Group  (CASWG),  as 
well  as  information  exchange  and  collaboration  in  a  variety 
of  areas  of  interest.  The  CASWG  is  developing  processes  to 
synchronize  planning,  execution,  and  reporting  among  all  cyber 
assessment  activities,  and  especially  those  supporting  exercise 
assessments. 

•  Sharing  of  information  and  expertise  with  the  Joint  Staff’s  Joint 
Deployable  Analysis  Team  continues  to  enhance  assessments. 
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The  partnership  collaborated  to  conduct  two  assessments  in 
FY12,  and  further  joint  assessments  are  planned  for  FY13. 

•  Collaborating  with  the  intelligence  community,  the  National 
Security  Agency,  and  the  Service  Information  Warfare  centers 
to  improve  the  portrayal  of  the  representative  cyber  threats 
during  exercises.  The  Defense  Intelligence  Agency  (D1A) 
made  significant  progress  in  defining  advanced  and  emerging 
methods  of  cyber  attack,  and  was  instrumental  in  mapping 
known  adversary  activities  to  the  threat  portrayals  for  several 
FY12  exercises. 

•  Working  with  the  Naval  Postgraduate  School  to  research 
and  develop  improved  capabilities  for  network  analyses. 


This  partnership  includes  the  design  and  development  of 
network  test  tools;  instrumentation;  training  resources 
and  test/ evaluation  methods;  analysis  of  compliance  and 
performance  findings  to  postulate  cause/effect  models  for  use 
in  simulation;  and  mapping  of  direct  operational  effects  arising 
from  network  performance  issues. 

•  Coordinating  with  the  Defense  Information  Systems 
Agency  (DISA)  to  improve  and  expand  the  assistance  and 
training  available  to  assessed  organizations,  including  the 
implementation  of  a  cyber-defense  training  and  assessment 
suite  at  several  CCMDs. 


FY1 2  ACTIVITIES 


In  FY12,  the  six  assessing  organizations  were  the  Army  Test 
and  Evaluation  Command;  the  Navy’s  Commander,  Operational 
Test  and  Evaluation  Force;  the  Marine  Corps  Operational 
Test  and  Evaluation  Activity;  the  Joint  Interoperability  Test 
Command;  the  Air  Force  Operational  Test  and  Evaluation 
Center;  and  the  Air  Force  688th  Information  Operations  Wing. 
These  6  assessing  organizations  completed  20  exercises  or  site 


assessments  under  the  IA  and  IOP  Assessment  Program,  and  2 
reduced  scope  assessments  at  sites  where  exercise  activity  was 
either  curtailed  or  cancelled.  These  assessments  included  13 
CCMD  and  6  Service  exercise  assessments  (see  Table  1).  Three 
assessments  involved  units  deployed  in  the  USCENTCOM  area 
of  responsibility. 


TABLE  1.  INFORMATION  ASSURANCE  AND  INTEROPERABILITY  EXERCISE  EVENTS  IN  FY12 


EXERCISE  AUTHORITY 

EXERCISE 

ASSESSMENT  AGENCY 

U.S.  Africa  Command 

Judicious  Response  201 2  (Exercise  Cancelled) 

ATEC 

U.S.  Central  Command 

AOR  Site  Assessment  #1 

ATEC 

AOR  Site  Assessment  #2 

ATEC 

AOR  Site  Assessment  #3 

ATEC 

U.S.  Cyber  Command 

Cyber  Flag  2012 

ATEC 

U.S.  European  Command 

Austere  Challenge  (Exercise  Cancelled) 

ATEC 

North  American  Aerospace  Defense 
Command  / 

U.S.  Northern  Command 

Vigilant  Shield  2012 

688  IOW 

Ardent  Sentry  2012 

688  IOW  &  AFOTEC 

Vibrant  Response  201 2 

JITC 

U.S.  Pacific  Command 

Terminal  Fury  2012 

COTF 

U.S.  Southern  Command 

PANAMAX  2012 

ATEC 

U.S.  Special  Operations  Command 

Emerald  Warrior  201 2 

ATEC 

U.S.  Strategic  Command 

Global  Lightning  2012 

JITC 

U.S. Transportation  Command 

Assessment  During  Operations 

JITC 

U.S.  Forces  Korea 

Key  Resolve  2012 

ATEC 

Ulchi  Freedom  Guardian  2012 

ATEC 

U.S.  Army 

Full  Scope  Exercise  2012-4 

ATEC 

U.S.  Navy 

Bold  Alligator  201 2 

COTF 

U.S.  Air  Force 

Angel  Thunder  2012 

JITC 

Red  Flag  2012-3 

688  IOW 

U.S.  Marine  Corps 

Ulchi  Freedom  Guardian  201 2  (III  MEF) 

MCOTEA 

Bold  Alligator  201 2 

MCOTEA 

AOR  -  Area  of  Responsibility  AFOTEC  -  Air  Force  Operational  Test  and  Evaluation  Center  ATEC  -  Army  Test  and  Evaluation  Command 

COTF  -  Commander,  Operational  Test  and  Evaluation  Force  IOW  -  Information  Operationswing  JITC  -  Joint  Interoperability  Test  Command 
MCOTEA  -  Marine  Corps  Operational  Test  and  Evaluation  Activity  MEF  -  Marine  Expeditionary  Force 
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Several  developments  in  FY12  confirm  increasing  emphasis 
across  the  DoD  to  prepare  to  train  and  operate  in  a  contested 
cyberspace  environment.  The  Chairman,  Joint  Chiefs  of  Staff 
(CJCS)  is  preparing  additional  guidance  to  amplify  the  Execute 
Order  (EXORD)  issued  in  FY11  to  increase  realistic  cyberspace 
conditions  in  training  exercises.  Threat  portrayal  improved 
during  assessed  training  exercises  but  with  limited  progress 
made  towards  implementing  EXORD  requirements.  The  overall 
number  of  instances  in  which  exercise  commanders  permitted 
cyber  effects  to  disrupt  operations  increased,  as  did  the  number  of 
sites  where  these  effects  were  demonstrated;  however,  the  overall 
effect  remains  low  due  to  constraints  imposed  upon  Red  Teams. 

The  Defense  Intelligence  Agency  (DIA)  Cyber  Threat 
Assessment  (CTA)  Office  continued  to  make  significant  progress 
in  defining  advanced  and  emerging  methods  of  cyber  attack,  and 
was  instrumental  in  mapping  known  adversary  activities  to  the 
threat  portrayals  for  several  FY 12  exercises.  For  example,  CTA 
threat  assessments  for  U.S.  Pacific  Command’s  (USPACOM) 
Terminal  Fury  2012  contributed  to  an  integrated  Red  Team 
employing  multiple  attack  vectors,  an  opposing  force  (OPFOR) 


with  a  cyber  cell  that  controlled  the  Red  Team  and  received 
exfiltrated  information,  and  some  of  the  most  realistic  cyber 
play  observed  to  date  in  an  exercise.  CTA  also  has  developed  a 
method  to  assess  the  shortfalls  between  the  postulated  threat  and 
the  threat  that  was  actually  present  in  training,  which  will  be  a 
key  metric  for  evaluating  implementation  of  the  CJCS  EXORD. 

To  enhance  the  IA  posture  of  acquisition  programs,  DOT&E 
continued  to  revise  and  refine  the  templates  and  process  for 
assessing  the  adequacy  of  1A  testing  in  acquisition  Test  and 
Evaluation  Master  Plans  and  test  plans.  These  templates 
facilitate  development  and  review  of  these  documents  to  ensure 
that  IA  is  adequately  addressed.  DOT&E  applied  the  templates 
and  new  process  to  the  Test  and  Evaluation  Master  Plans  for  34 
systems,  the  operational  test  plans  of  13  systems,  and  related 
test  documents  of  8  systems.  Additionally,  DOT&E  IA  experts 
specifically  observed  IA  tests  and  reviewed  data  for  the  following 
three  systems  after  previously  reviewing  test  documentation: 

•  Patriot  Missile  (PAC-3) 

•  U.S.  Navy  dry  cargo  ship  (T-AKE) 

•  U.S.  Army  Apache  Block  III  helicopter 


FINDINGS,  TRENDS,  AND  ANALYSIS 


Interoperability 

The  FY12  assessments  found  that  interoperability  issues 
encountered  by  the  exercise  training  audience  largely  hindered, 
but  ultimately  did  not  prevent  mission  accomplishment.  This  was 
primarily  because  operators  developed  and  executed  effective 
workarounds.  The  workarounds  increased  operator  workload, 
and  often  degraded  the  efficiency  of  completing  tasks,  or 
degraded  timeliness  and  accuracy  of  the  information  generated. 

Operators  frequently  view  interoperability  problems  within 
systems  architectures  as  maintenance  or  design  issues  beyond  the 
control  of  local  authorities.  Therefore,  many  of  these  problems 
are  not  reported,  documented,  or  remediated.  More  often,  local 
users  will  develop  practices  and  techniques  to  work  around  the 
lack  of  a  desired/designed  automated  function.  Workarounds 
include  such  techniques  as: 

•  Manual  transcription  of  data  from  one  system  to  another, 
introducing  transcription  errors 

•  Data  transfer  between  systems  via  portable  media,  thereby 
opening  both  systems  to  outside  malware  intrusion 

•  System  re -boot/re-set  to  trigger  update  routines,  usually 
resulting  in  increased  delay  and  latency  of  operational  data 

System-to-system  interoperability  problems  remain  largely 
unreported.  Over  the  last  two  years,  slightly  less  than  one-quarter 
of  all  systems  observed  during  exercise  assessments  had  been 
fully  certified  for  interoperability.  Of  those  systems,  only 
two-fifths  have  ever  been  previously  certified,  indicating  that 
almost  half  of  the  exercise  systems  have  lapsed  in  certification 
or  been  replaced  by  uncertified  software  versions.  Configuration 
management  and  documentation  of  observed  systems  (certified  or 
not)  were  reported  by  the  system  operators  to  the  operational  test 
observers  as  satisfactory  for  9  of  every  10  systems.  Operators 


cited  system  reliability  as  a  problem  in  almost  one-third  of  all 
systems  reviewed  in  FY12,  an  increase  over  previous  years. 
Several  of  the  findings  either  reported  or  under  research  by 
DOT&E  involve  interoperability  shortfalls,  including: 

•  AOC  Interoperability  -  software  baseline  and  interoperability 
certification  in  the  Air  and  Space  Operations  Centers  lacks 
centralized  configuration  management  and  control.  As  a 
result,  the  Air  and  Space  Operations  Centers  do  not  have 
standard  software,  and  frequently  employ  locally-produced 
middleware  to  accommodate  system-to-system  interoperation. 
F  urthermore,  the  version  of  the  Global  Command  and  Control 
System  (GCCS)  in  use  at  all  of  the  AOCs  had  not  been  fully 
tested  or  certified  for  operational  use.  (Note:  since  the  release 
of  this  finding,  the  testing  and  certification  of  the  most  recent 
update  for  GCCS-Joint  is  in  progress,  which  includes  AOC 
operational  support.) 

•  Third  Party  Patching  -  DoD  uses  a  large  number  of 
commercial  software  suites,  ranging  from  the  baseline 
Windows®  Operating  System  on  most  desktop  computers, 
Adobe  file  readers,  JAVA  script,  and  other  commonly  available 
commercial  administrative  and  business  software.  DoD  does 
not  have  a  means  of  central  management  for  updates  to  these 
commercial  applications,  requiring  local  network  authorities 
to  download  commercial  patches  and  updates,  test,  and 
implement  them  individually. 

•  Surveillance  Radar  Systems  -  A  wide-area  surveillance 
radar  system  observed  during  one  exercise  was  found 
to  potentially  allow  control  of  the  sensor  from  multiple 
workstations/roles/ accounts  within  the  command  and  control 
software  that  accesses  the  radar  -  essentially  preventing  a 
stable  configuration  during  operations. 
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Information  Assurance 

Red  Teams  reported  increased  difficulty  in  penetrating  network 
defenses;  however,  results  show  that  with  sufficient  time,  Red 
Teams  routinely  penetrated  networks  and  systems  with  few 
exceptions.  Detection  rates  of  network  intrusions  remained 
low,  and  the  ability  of  network  defenders  to  detect  subsequent 
exploitations  of  information  was  minimal. 

The  CJCS  EXORD  of  February  2011  to  Incorporate  Realistic 
Cyberspace  Conditions  into  Major  DoD  Exercises  directs 
more  realistic  cyber  adversary  portrayals  in  all  major  CCMD 
and  Service-level  exercises.  There  is  little  evidence  that  the 
milestones  cited  in  the  EXORD  (such  as  identification  of  critical 
mission  tasks  and  systems)  have  been  completed. 

The  level  of  threats  portrayed  in  assessed  exercises  in  FY12 
(see  Figure  1)  remained  similar  to  previous  years,  with  a 
modest  increase  in  both  high-level  portrayals  and  exercises 
in  which  no  threat  was  portrayed  (usually  onsite/non-exercise 
assessments  without  Red  Teams).  While  exercise  commanders 
permitted  degraded  network  operations  on  almost  twice  as  many 
unclassified  network  sites  than  the  previous  year,  the  instances 
of  degraded  performance  on  classified  networks  declined 
slightly.  In  FY12,  a  quarter  of  all  Red  Team  activities  were 
directly  disruptive  to  networks  assessed,  a  step  forward  in  the 
implementation  of  the  EXORD  guidance.  However,  in  cases 
where  adversary  teams  portrayed  higher-level  threats,  exercise 
training  audiences  were  unable  to  either  develop  or  demonstrate 
advanced  mitigation  or  tactics  in  the  face  of  these  threats.  As 
a  result,  the  exercise  participants'  defensive  actions  were  not 
well-matched  to  the  threats  portrayed,  and  sometimes  exacerbated 
the  negative  effects  of  the  cyber  threat. 


Figure  1:  Distribution  of  Threat  Depictions 


Threat  Levels  Depicted 

Figure  1:  Distribution  of  threat  portrayals  in  assessed  exercises.  The 
majority  of  threats  portrayed  in  FY12  were  low-to-medium  capability, 
and  occurred  less  often  than  in  FY1 1 .  A  modest  increase  in  high-level 
threat  portrayals  was  seen  at  a  limited  number  of  exercise  sites. 

Exercise  personnel  at  times  misinterpreted  the  cyber  effects 
from  these  more  aggressive  and  disruptive  threat  portrayals 
as  arising  from  non-adversary  causes  such  as  maintenance 


shortfalls,  system  performance  problems,  or  even  as  artificialities 
within  the  exercise  construct.  As  a  result,  exercise  participants 
either  ignored  or  otherwise  did  not  report  significant  network 
events,  essentially  denying  network  defenders  and  leadership 
critical  knowledge  of  the  network  status.  Additionally,  exercise 
participants  perceived  the  attribution  process  (confirm  whether 
Red  Teams  caused  the  effect)  as  cumbersome  and  slow,  and  in 
several  cases,  simply  ignored  this  process,  further  detracting  from 
the  ability  to  develop  a  concise  and  accurate  view  of  the  networks 
under  observation. 


Most  network  compliance  attributes  continue  to  gradually 
improve,  indicating  greater  compliance  with  basic  standards  (see 
Figure  2).  Network  boundary  defense  compliance  continued 
to  improve,  including  the  presence  of  host-based  intrusion 
detection  tools,  improved  configuration  management  of  networks 
and  security  tools,  and  the  overall  infrastructure  supporting 
the  operational  networks.  Physical  environment,  enclave 
boundary  protections,  and  incident  management  is  improving. 
The  effective  use  of  host-based  intrusion  detection  systems,  for 
example,  is  increasing. 

The  ongoing  fielding  of  the  Host-Based  Security  System 
(HBSS)  is  improving  compliance  with  having  local  network 
protection  and  intrusion  detection;  however,  the  majority  of 
HBSS  suites  DOT&E  observed  were  found  to  be  incorrectly 
or  ineffectively  configured.  Enforcement  of  configuration 
standards;  the  deliberate  planning  for  incident  responses;  and 
critical  network  infrastructure  practices  to  include  having  backup 
components,  supplies,  and  spares  continue  to  improve.  In  at 
least  one  assessment,  a  strict  enforcement  of  these  basic  network 
requirements  resulted  in  measurably  reduced  Red  Team  success. 
Efforts  are  also  underway  at  selected  CCMDs  to  document  and 
develop  Computer  Network  Defense  playbooks  as  training  and 
operational  tools. 
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Figure  2:  Site  Compliance 
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Figure  2:  Six-year  trend  in  compliance  standards  as  measured  during 
exercise  assessments.  Physical  environment  and  enclave  protection 
standards  have  improved  steadily  since  FY07,  but  progress  in 
operational  network  continuity,  configuration  and  design  standards, 
and  vulnerability/incident  management  remain  gradual. 
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DOT&E  observed  reduced  rates  of  compliance  in  the  use  of 
software  and  hardware  backups  and  key  practices  such  as  port 
and  protocol  protections,  reliable  software  baselines,  remediation 
of  known  vulnerabilities,  and  effective  use  of  system  audit  logs. 
Exercise  authorities  rarely  shift  to  alternate  sites  or  systems. 

Most  audit  logs  did  not  meet  the  minimum  requirements 
specified,  and  the  identification  and  remediation  of  known 
vulnerabilities  has  also  declined  over  the  last  two  years.  DOT&E 
also  observed  that  the  experience  and  formal  training  levels  for 
network  defenders,  which  increased  steadily  over  the  last  three 
years,  showed  a  large  influx  of  new  or  inexperienced  personnel  in 
FY12. 

Mission  Assurance 

During  the  FY12  assessments,  the  operational  testers  analyzed 
the  1A  and  IOP  findings  to  characterize  the  operational  effects, 
or  potential  operational  effects,  on  specific  missions.  Although 
cyber-adversary  activities  posed  a  high  risk  to  critical  operations, 
exercise  authorities  seldom  permitted  disruptions  to  be  fully 
exercised;  the  priority  to  achieve  other  exercise  training 


objectives  remains  at  odds  with  exercising  in  an  environment 
with  representative  cyber  adversaries  and  consequently  degraded 
systems.  In  those  exercises  where  operational  effects  were 
permitted,  the  mission  impacts  included: 

•  Disclosure  of  friendly  force  locations  and  activities  to  the 
opposition  force,  resulting  in  fewer  adversary  losses 

•  Denial  of  critical  network  capabilities  during  periods  of  high 
operational  tempo 

•  Delays  in  the  delivery  of  operational  data 

Despite  the  few  permitted  and  observable  impacts  to  actual 
missions,  DOT&E  analysis  of  the  vulnerabilities,  intrusions, 
and  compliance  trends  clearly  indicates  high-risk  to  operational 
tasks  and  Combatant  Commander  operational  missions. 

DOT&E  analysis  indicates  that  without  the  development  of 
defensive  tactics  commensurate  with  the  sophistication  of  our 
adversaries,  large-scale  compromise  or  loss  of  operational  data 
and  operational  systems  during  high-tempo  operations  cannot  be 
discounted. 


REPORTS 


Each  of  these  assessments  resulted  in  a  specific  report  for  the 
Exercise  Authority  (CCMD  or  Service)  detailing  problems 
found  during  the  exercise  and  detailed  observations  and 
recommendations.  In  addition  to  these,  DOT&E  published 
five  memoranda  of  findings  and  initiated  research  on  nine 
additional  areas  of  concern  in  FY12.  Finding  memoranda  detail 
specific  shortfalls  and  vulnerabilities  that  have  the  potential  to 
significantly  degrade  operations  and  warrant  senior  leadership 
attention.  DOT&E  identified  the  shortfalls  and  vulnerabilities  to 
the  responsible  leadership.  Service  and  CCMDs  provided  replies 
to  DOT&E  detailing  mitigation  efforts,  which  then  are  subject  to 
subsequent  re-evaluation  and  validation  in  future  assessments. 
During  the  fiscal  year,  where  observable,  DOT&E  reviewed  or 
validated  in  the  field  solutions  to  prior  findings.  New  findings 
released  or  researched  in  FY12  are  listed  below. 

Released  in  FY12: 

•  Air  Operations  Center  (AOC)  Interoperability  (released 
November  201 1)  -  documented  the  lack  of  a  consistent 
software  baseline  and  interoperability  certification  in  the  Air 
and  Space  Operations  Centers 

•  Virtual  Secure  Enclaves  (released  December 

2011)  -  documented  a  promising  network  security  experiment 
at  USPACOM 

•  Third  Party  Patching  (released  January  2012)  -  documented 
a  lack  of  central  management  for  security  patches  on 
commercial  software  in  use  within  DoD  networks 

•  Active  Directory  Pass-the-Flash  (released  March 

2012)  -  documented  a  classified  investigation  into  a  common 
hacker  technique 

•  Assessment  of  DoD  1A  during  Major  CCMD  and  Service 
Exercises  (published  April  2012)  -  documented  a  detailed 


follow-up  to  the  FY 1 1  Annual  Report,  specifically  addressing 
classified  IA  issues 

Research  Initiated  in  FY12: 

•  F1BSS  discrepancies  in  asset  management  (initiated  March 
2012  and  released  October  2012)  -  investigating  a  potential 
common  misconfiguration  of  the  system  that  causes  inaccurate 
or  inconsistent  results 

•  Event  attribution  (initiated  May  2012)  -  investigating  the 
manner  in  which  events  detected  during  an  exercise  are 
attributed  to  either  Red  Team  activity  or  actual  cyber  incidents 

•  Shipboard  Systems  (initiated  July  2012)  -  investigating  a 
possible  vulnerability  to  afloat  systems 

•  Physical  Intrusion  Devices  (initiated  July  2012)  -  investigating 
the  use  of  a  commonly  available  hacker  tool 

•  Password  shortfalls  (initiated  July  2012)  -  investigating 
common  password  errors  exploited  by  Red  Teams 

•  Unsecured  chat  systems  (initiated  July  2012)  -  investigating 
the  operational  effects  of  using  collaboration  tools  that  can  be 
easily  intruded/exploited 

•  Phishing  and  misuse  of  secure  socket  technology  (initiated 
July  2012)  -  investigating  the  operational  effects  of  two 
common  hacker  techniques 

•  Physical  Security  (initiated  July  2012)  -  investigating  multiple 
instances  and  causes  of  failures  to  physically  protect  network 
resources  and  points  of  access 

•  Surveillance  Radar  Systems  (initiated  September 
2012)  -  documenting  a  possible  control-of-radar 
interoperability  problem 
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FY1 3  PLANS  AND  GOALS 


DOT&E’s  goal  is  to  complete  at  least  one  IOP  and  one  LA 
assessment  of  each  CCMD  and  Service  during  the  fiscal  year, 
with  15  CCMD  and  Service  exercises  already  identified  for 
FY13  (see  Table  2).  One  of  the  planned  FY13  assessments  will 
involve  units  already  deployed  to  the  U.  S.  Central  Command 
(USCENTCOM)  areas. 

The  FY13  IA/IOP  Assessment  Program  will  focus  on  the 
following  goals: 

•  Supporting  and  monitoring  the  three-year  implementation  of 
the  CJCS  EXORD,  and  continuing  to  improve  the  realism  of 
portrayed  cyber  threats  during  assessments 


•  Developing  and  implementing  additional  improvements  to  the 
methods  for  gathering  and  assessing  the  effects  on  operational 
missions 

•  Increasing  coordination  with  USCYBERCOM,  DISA,  DoD 
CIO,  and  other  agencies  in  the  scheduling  and  conduct  of 
assessments 

•  Continuing  to  expand  the  use  of  the  DoD  Joint  Information 
Operations  Range  (JIOR)  and  other  range/test  facilities  in 
support  of  exercise  assessments 

•  Continuing  to  refine  the  mission  assurance  analysis  afforded 
by  the  IA  and  IOP  findings 


TABLE  2.  INFORMATION  ASSURANCE  AND  INTEROPERABILITY  EXERCISE  EVENTS  PROPOSED  FOR  FY13 

EXERCISE  AUTHORITY 

EXERCISE 

ASSIGNMENT  AGENCY 

U.S.  Africa  Command 

Judicious  Response  201 3 

ATEC 

U.S.  Central  Command 

AOR  Site  Assessment 

ATEC 

Internal  Look  201 3 

ATEC 

U.S.  Cyber  Command 

Cyber  Flag  2013 

ATEC 

U.S.  European  Command 

AOR  Site  Assessment 

ATEC 

North  American  Aerospace  Defense 
Command  /  U.S.  Northern  Command 

Vigilant  Shield  2013 

AFOTEC 

U.S.  Pacific  Command 

Terminal  Fury  2013 

COTF 

U.S.  Southern  Command 

Integrated  Advance  2013 

ATEC 

U.S.  Special  Operations  Command 

Emerald  Warrior  201 3 

ATEC 

U.S.  Strategic  Command 

Global  Lightning  2013 

JITC 

U.S. Transportation  Command 

Turbo  Challenge  2013 

JITC 

U.S.  Army 

Warfighter  13-4 

ATEC 

U.S.  Navy 

Aircraft  Carrier  Battle  Group  (CVBG)  Assessment 

COTF 

U.S.  Air  Force 

Blue  Flag  2013 

AFOTEC 

U.S.  Marine  Corps 

II  Marine  Expeditionary  Force 

MCOTEA 

AOR  -  Area  of  Responsibility  AFOTEC  -  Air  Force  Operational  Test  and  Evaluation  Center  ATEC  -  Army  Test  and  Evaluation  Command 

COTF  -  Commander,  Operational  Test  and  Evaluation  Force  JITC  -  Joint  Interoperability  Test  Command 

MCOTEA  -  Marine  Corps  Operational  Test  and  Evaluation  Activity 

INFRASTRUCTURE  OBSERVATIONS 


While  exercise  commanders  permitted  degraded  network 
operations  on  almost  twice  as  many  unclassified  network  sites 
than  the  previous  year,  the  instances  of  degraded  performance  on 
classified  networks  declined  slightly.  Exercise  authorities  remain 
cautious  about  permitting  advanced  threat  depictions  or  advanced 
network  effects  that  may  endanger  other  exercise  objectives,  or 
be  inappropriate  for  conduct  on  live  networks. 

The  use  of  cyber  ranges  and  laboratories  increased  in  FY12, 
with  four  exercises  incorporating  ranges  to  support  exercise 
conduct:  RED  FLAG  2012,  CYBERFLAG  2012,  TERMINAL 
FURY  2012,  and  WARFIGHTER  2012-4.  For  RED  FLAG 
and  CYBERFLAG,  the  use  of  the  cyber  range  was  integral 
to  the  exercise,  whereas  during  the  TERMINAL  FURY  and 


WARFIGHTER  exercises,  cyber  range  use  supplemented 
and  enhanced  the  training  scenarios  but  was  not  central  to  the 
exercises.  In  all  four  instances,  the  use  of  the  ranges  permitted 
more  advanced  threat  and  network  activities. 

The  CJCS  EXORD  of  February  2011  directed  more  realistic 
cyber  adversary  portrayals  in  all  major  CCMD  and  Service-level 
exercises,  but  did  not  specify  all  of  the  necessary  resources 
to  accomplish  this  tasking.  Expanded  use  of  range  facilities 
has  been  demonstrated  to  both  enhance  and  expand  the  ability 
to  depict  wider  varieties  of  cyber  activities.  Furthermore, 
many  DoD  networks  have  transitioned  from  direct  CCMD 
management  and  oversight  to  “ownership”  by  consolidated  cyber 
service  providers  or  Service  component  cyber  commanders. 
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The  networks  supporting  USAFRICOM  and  USEUCOM, 
for  example,  are  now  consolidated  under  the  Joint  Enterprise 
Network  (JEN)  for  the  theater  and  under  the  control  and 
management  by  the  Army  Signal  Brigade  in  that  theater.  A 
number  of  Air  Force  networks  are  similarly  consolidating. 

All  of  these  consolidations  are  consistent  with  the  Department’s 
plans  for  a  Joint  Information  Enterprise,  but  this  has  further 
complicated  the  tasks  of  planning  and  executing  realistic 
assessments.  In  recent  exercises,  the  assessing  agency  either 
experienced  critical  delays  or  was  unable  to  obtain  approved 
ground  rides,  authorizations,  or  support  for  cyber  adversary 
activities  during  the  exercise.  This  was  largely  due  to  the  cyber 
component’s  inability  to  support  the  additional  activities  required 
by  the  exercise,  or  the  lack  of  sufficient  agreements  with  the 
supported  commander  to  make  such  commitments  on  behalf  of 
the  Combatant  Commander.  As  DoD  continues  to  consolidate 
cyber  resources,  it  will  be  critical  for  these  agencies  to  control 
sufficient  resources  to  support  exercises  to  the  degree  required  by 
the  EXORD.  Additionally,  the  demand  for  “offline”  capabilities, 
such  as  training,  experimentation,  development,  and  test  ranges 
will  increase. 

DOT&E  continues  to  support  the  development  of  methods 
and  environments  to  exercise  and  assess  advanced  actions  on 


appropriate  closed-loop  cyber  ranges.  CCMDs  used  cyber 
ranges  such  as  the  JIOR  in  four  assessed  exercise  venues, 
and  emphasis  will  continue  for  increasing  the  integration  and 
operational  realism  of  JIOR  events  associated  with  DOT&E’s 
IA/IOP  assessments  in  FY13.  DOT&E  sponsored  a  distributed 
cyber-range  experiment  in  July  2012,  where  the  JIOR  was  used 
to  connect  the  National  Cyber  Range  (NCR)  with  other  cyber 
labs,  targets,  and  attackers.  NCR  capabilities  offer  substantial 
increases  in  network  scaling  and  substantial  reductions  in  the 
time  required  for  cyber  research,  development,  training,  and 
testing. 

At  DOT&E's  initiative  to  enhance  the  operational  realism  and 
threat  portrayal  in  exercises  and  range  environments,  DoD 
championed  investments  to  mature  the  environments  and 
capabilities  needed  for  testing  and  training  with  advanced 
cyber  adversaries.  The  need  for  this  capability  is  highlighted 
by  the  findings  contained  in  the  DOT&E  classified  report  dated 
April  2012.  DOT&E  recommended  integrating  four  facilities 
into  an  enterprise  cyber  range  to  speed  implementation  of  the 
CJCS  EXORD,  as  well  as  to  meet  Section  933  requirements  for 
infrastructure  to  support  the  rapid  acquisition  of  cyber  warfare 
capabilities. 
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Test  and  Evaluation  Resources 


Title  10,  U.S.  Code  requires  DOT&E  to  assess  the  adequacy 
of  operational  and  live  fire  testing  conducted  for  programs 
under  oversight  and  to  include  comments  and  recommendations 
on  resources  and  facilities  available  for  operational  test  and 
evaluation  and  levels  of  funding  made  available  for  operational 
test  and  evaluation  activities.  DOT&E  monitors  and  reviews 
DoD  and  Service-level  strategic  plans,  investment  programs, 
and  resource  management  decisions  to  ensure  capabilities 
necessary  for  realistic  operational  tests  are  supported.  This 
report  addresses  the  major  areas  of  concern  in  testing  current 
systems  and  discusses  both  resource  needs  and  significant 
issues  for  operational  and  live  fire  testing.  Specific  capabilities 
and  test  resource  areas  of  concern  include: 

•  General  Test  Infrastructure 

•  Next-Generation  Electronic  Warfare  Environment  Generator 
(NEWEG) 

•  Integrated  Technical  Evaluation  and  Analysis  of  Multiple 
Sources  (ITEAMS) 


•  Advanced  Electronic  Warfare  (EW)  Test  Resources 

•  Cyber  Warfare 

•  Explosive  Surrogate  for  Use  in  Live  Fire  Test  and 
Evaluation  (LFT&E) 

•  Tactical  Engagement  Simulation  (TES)  and  Real  Time 
Casualty  Assessment  (RTCA) 

•  Joint  Urban  Test  Capability  (JUTC) 

•  F ifth-Generation  Aerial  Target 

•  Automated  Test  Capabilities  for  Software-Based  Systems 

•  Steerable  Antenna  for  GQM- 1 63  A  Supersonic  Target 
Testing 

•  Additional  Electronic  Warfare  (EW)  Simulator  Units  for 
Surface  EW  Improvement  Program  (SEWIP)  Block  2 

•  Continuing  Radio  Frequency  Spectrum  Concerns 

•  Renewable  Energy  Infrastructure  Impact  on  Operational 
Testing 


General  Test  Infrastructure 

The  DoD  budget  remains  under  severe  fiscal  pressure,  and  the 
DoD  faces  significant  budget  uncertainty  due  to  legislation 
limiting  Federal  spending.  In  addition  to  uncertain  budgets  and 
funding  challenges,  the  test  infrastructure  faces  technological 
and  policy  challenges  and  risks  in  maintaining  capabilities  to 
test  and  evaluate  the  effectiveness,  suitability,  survivability, 
and/or  lethality  of  current  and  future  defense  systems.  The  Test 
Resources  Management  Center  (TRMC)  identified  a  variety 
of  near-term  test  infrastructure  needs  in  developing  the  2012 
Strategic  Plan  for  DoD  T&E  Resources.  Some  of  these  needs 
include: 

•  Addressing  near-term  maintenance,  sustainment,  and 
modernization  needs  of  T&E  facilities  across  the  Services  due 
to  obsolescence  and  equipment  deterioration 

•  Managing  the  current  workforce  while  shaping  future 
workforce  requirements  to  meet  the  sophisticated  T&E  and 
acquisition  challenges  brought  about  by  emerging  technology 

•  Developing  an  investment  and  operational  strategy  to  produce 
unmanned  and  autonomous  systems  test  capability  in  the  air, 
land,  and  maritime  domains 

•  Continuing  initial  efforts  to  develop  a  cyberspace  test 
infrastructure  capability  that  provides  blue,  red,  and  gray 
environments  with  representative  threats  to  offer  both 
defensive  and  offensive  cyber  operations 

The  test  infrastructure  provides  critical  support  for  operational 
and  live  fire  testing,  and  DOT&E  is  working  closely  with  the 
Deputy  Assistant  Secretary  of  Defense,  Developmental  Test  and 


Evaluation  (DT&E)/Director,  TRMC  to  ensure  that  the  DoD 
retains  sufficient  core  capabilities  to  conduct  realistic  testing. 
DOT&E  has  also  engaged  in  the  DoD  budget  process  to  address 
problems  with  electronic  warfare  threat  simulators,  the  cyber 
range,  and  body  armor  testing. 

Savings  in  Test  Infrastructure. 

Given  the  current  and  projected  fiscal  environment,  DOT&E 
understands  fully  the  need  to  seek  savings  in  all  areas  of  the 
Department’s  activities  and  infrastructure,  including  testing  and 
test  infrastructure.  The  need  for  cost-effective  testing  is  a  key 
reason  DOT&E  continues  to  require  that  test  plans  incorporate 
the  use  of  rigorous  statistical  methods  for  determining  the  scope 
and  breadth  of  operational  testing  and  for  evaluating  the  data 
produced  by  that  testing.  During  the  past  year,  the  developmental 
test  office  has  attempted  to  identify  savings  that  could  be 
accrued  within  the  Department’s  test  infrastructure,  as  well  as 
by  adopting  revised  testing  practices,  and  has  produced  a  draft 
“Comprehensive  Review  of  Test  and  Evaluation  Infrastructure.” 
Although  adoption  of  some  of  the  report’s  proposals  for  revised 
practices  would  be  beneficial,  the  draft  report  provided  to 
DOT&E  for  review  has  serious  shortcomings.  Savings  taken 
within  any  of  the  Department’s  activities  or  infrastructure  must 
be  based  on  actions  that  are  clearly  defined  and  can,  therefore, 
actually  be  taken;  must  be  capable  of  assignment  to  specific 
elements  of  the  defense  budget;  and,  must  be  based  on  credible 
estimates.  Unfortunately,  the  draft  report  reviewed  by  DOT&E 
satisfies  none  of  these  important  prerequisites. 
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Next-Generation  Electronic  Warfare  Environment  Generator 
(NEWEG) 

Electronic  Warfare  (EW)  threat  simulation  capabilities  must  meet 
ever-expanding  requirements  for  future  strategic,  tactical,  and 
support  aircraft  with  electronic  combat  support  systems.  These 
requirements  are  the  result  of  the  increasing  sophistication  and 
capabilities  of  threat  radars  and  the  increasing  detection  and 
processing  capabilities  of  EW  systems,  as  well  as  the  need  for 
higher-fidelity  simulation  capability  to  provide  closer  correlation 
between  laboratory  testing  and  flight  testing.  Such  a  simulation 
capability  would  reduce  overall  test  costs  and  allow  greater 
confidence  in  laboratory  results.  To  meet  these  emerging  EW 
requirements,  a  new  generation  of  E  W  simulators  must  make 
use  of  state-of-the-art  technologies,  including  modular  and 
scalable  architectures,  high-speed  processing,  and  integrated 
radio  frequency  (RF)  subassemblies.  The  T&E  community 
requires  advanced  software  models  to  represent  platform  motion, 
direction-finding,  modulation,  and  environmental  conditions. 
There  is  currently  no  capability  to  characterize  multiple 
interdependent  jammers  that  simultaneously  jam  different  targets 
in  different  bands  in  different  locations  in  space. 

The  NEWEG  project  will  use  a  state-of-the-art,  high-fidelity, 
modular,  scalable  and  reconfigurable  EW  environment  generator 
and  a  dynamic  multi-beam  characterization  capability  for 
current  and  future  EW  systems  testing.  It  will  also  establish 
commonality  among  DoD  stimulators.  NEWEG  will  support 
both  developmental  and  operational  testing  and  is  intended 
to  satisfy  shortcomings  at  Modeling  and  Simulation  and 
Hardware-in-the-Loop  Labs,  Installed  Systems  Integration 
Facilities,  and  Open-Air  Ranges.  The  Tri-Service  Electronic 
Warfare  Test  Capability  Study  identified  these  shortcomings  in 
August  2010. 

The  technical  objective  of  NEWEG  is  to  evolve  the 
state-of-the-art  in  EW  simulation  and  stimulation  technology 
into  much  higher-fidelity  threat  signal  simulation.  Additional 
project  benefits  include  establishing  commonality  between  DoD 
stimulators  (leading  to  improved  test  repeatability)  and  continuity 
between  ranges  and  facilities  (resulting  in  reduced  preparation 
time  and  cost).  NEWEG  is  intended  to: 

•  Incorporate  an  open-architecture  design  facilitating 
information  sharing  between  EW  testers 

•  Incorporate  dynamic  jammer  recording  and  analysis 

•  Include  a  limited  closed-loop  threat  response  capability  for 
Electronic  Attack  reactivity  evaluation 

•  Allow  playback  of  collected  Intelligence/SEI  (Specific  Emitter 
Identification)  waveforms 

•  Incorporate  an  integrated  dynamic  motion-based  RF 
receive/transmit  and  analysis  subsystem  into  the  Advanced 
Systems  Integration  Laboratory  (ASIL)  at  Patuxent  River, 
Maryland 

This  capability  is  required  for  adequate  operational  testing  of 
F-35  Block  3  in  FY17  and  beyond.  NEWEG  development 
is  estimated  to  cost  $23.9  Million  to  meet  the  threshold  Key 
Performance  Parameters  (KPPs)  for  the  system  and  $33.9  Million 
to  achieve  the  objective  KPPs. 


Integrated  Technical  Evaluation  and  Analysis  of  Multiple 
Sources  (ITEAMS) 

After  the  fall  of  the  Soviet  Union,  U.S.  testers  were  able  to 
acquire  foreign  military  weapon  systems  for  testing  U.S.  weapon 
systems.  The  emergence  of  new  potential  adversaries  in  the  past 
two  decades  has  created  a  threat  situation  in  which  foreign  assets 
are  not  available  for  exploitation  or  testing.  The  Threat  Systems 
Program  (TSP)  supports  U.S.  Service  members  by  providing 
threat  intelligence  to  ensure  operational  and  developmental 
testing  occurs  against  realistic  threat  representations.  TSP 
is  a  partnership  of  the  intelligence,  operational  testing,  and 
acquisition  communities  in  the  DoD.  Under  a  memorandum 
of  agreement  between  DOT&E  and  the  Defense  Intelligence 
Agency  (DIA),  the  Test  and  Evaluation  Threat  Resource  Activity 
(TETRA),  within  the  Missile  and  Space  Intelligence  Center, 
executes  the  TSP  and  provides  ongoing  intelligence  analysis 
and  support  for  DOT&E  threat  resources  while  managing  and 
overseeing  a  DOT&E  investment  program  for  the  development  of 
threat  resources. 

TSP  pioneered  the  use  of  intelligence  deep-dives,  going  beyond 
the  normal  intelligence  mission  to  perform  intelligence  research 
and  analysis  necessary  to  develop  threat  simulators.  These 
efforts,  known  as  Integrated  Technical  Evaluation  and  Analysis  of 
Multiple  Sources  (ITEAMS),  can  result  in  threat  representations 
such  as  models  and  simulations  or  produce  blue -print  designs  for 
constructing  open-air  operational  test  assets,  such  as  hardware 
solutions  representing  a  threat  capability  or  function. 

During  FY12,  TSP  completed  ITEAMS  projects  on  land  and  sea 
threats.  A  scientific  and  technical  intelligence  staff  translated 
all  source  technical  intelligence  on  a  widely  proliferated  battle 
management  and  command,  control,  communication,  and 
computer  system  into  a  model  to  support  test  and  evaluation. 

The  model  is  intended  to  provide  real-time  man-in-the-loop 
operations  in  an  electronic  attack  environment.  Two  projects 
starting  in  FY12  will  address  medium-range  surface-to-air  missile 
threats  from  two  different  adversaries,  leveraging  previous 
intelligence  collection  and  analysis  on  these  systems  and  their 
predecessors.  These  efforts  are  in  addition  to  recently  completed 
ITEAMS  of  threat  Advanced  Air  Defense  Systems.  In  FY 12, 
DOT&E  authorized  funding  of  10  new  ITEAMS  to  address 
critical  operational  test  needs  for  2014  and  beyond. 

Advanced  EWTest  Resources 

In  February  2012,  DOT&E  identified  shortfalls  in  EW  test 
resources  that  prevent  development,  testing,  and  timely  fielding 
of  U.S.  systems  capable  of  operating  successfully  against 
threats  that  currently  exist,  are  proliferating,  and  are  undergoing 
an  accelerating  pace  of  significant  upgrades.  Subsequently, 
DOT&E  identified  the  need  for  approximately  $495  Million  in 
funding  from  FY13-18  to  address  these  shortfalls  and  assure 
the  needed  test  resources  would  be  available  in  time  to  support 
developmental  and  operational  testing  of  systems  including  the 
Joint  Strike  Fighter.  The  DOT&E  recommendations  included 
accelerating  the  NEWEG  program’s  production  of  high  fidelity 
signal  generators,  upgrading  the  government  anechoic  chambers 
with  adequate  numbers  of  signal  generators  from  the  NEWEG 
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program  for  realistic  threat  density,  upgrading  the  Joint  Strike 
Fighter  mission  data  file  reprogramming  lab  to  include  realistic 
threats  in  realistic  numbers,  providing  ITEAMS  products  needed 
to  guide  threat  simulations,  and  developing  a  combination  of 
open  and  closed-loop  threat  simulators  in  the  numbers  needed  for 
operationally  realistic  open-air  range  testing. 

DOT&E  participated  in  a  “tiger  team”  assigned  by  USD(AT&L) 
to  review  the  shortfalls  identified  by  DOT&E,  which  concurred 
with  DOT&E ’s  conclusions  and  recommended  additional 
enhancements.  The  combination  of  improved  government-owned 
anechoic  chambers  and  new  open-air  range  test  assets  is  needed 
to  evaluate  the  advanced  capabilities  under  development  in  the 
Joint  Strike  Fighter,  F-22  Increment  3.2  A/B,  B-2  Defensive 
Management  System,  Long-Range  Strike  Bomber,  Next 
Generation  Jammer  for  the  EA-18G,  Integrated  Defensive 
Electronic  Countermeasures  upgrades,  as  well  as  several  other 
programs.  DOT&E  is  pursuing,  along  with  the  TRMC,  the 
necessary  actions  to  acquire  the  test  assets  and  integrate  them  into 
operational  test  plans.  Absent  these  test  resources,  development 
and  adequate,  realistic  testing  of  the  systems  cited  above  will  not 
be  possible. 

Cyber  Warfare 

The  capacity  to  assess  realistically  advanced  cyber  warfighting 
capabilities  must  be  increased  to  keep  pace  with  heightened 
demand  for  those  capabilities,  advancing  technologies,  and  the 
growing  cyber  threat.  In  February  2011,  the  Chairman  of  the 
Joint  Chiefs  of  Staff  issued  a  memorandum  directing  that  all 
major  exercises  include  realistic  cyber  adversary  elements  as  a 
training  objective.  To  comply  with  this  order,  more  non-major 
exercises  are  needed,  and  these  exercises  must  include  realistic 
cyber  adversaries.  The  Joint  Information  Operations  Range 
offers  a  multi-level  security  environment  to  integrate  and  conduct 
simultaneous  cyber  activities. 

DOT&E  identified  a  $90  Million  need  over  the  Future  Years 
Defense  Program  (FYDP)  to  upgrade  range  operations  and 
capacity  to  conduct  additional  events,  handle  larger  amounts  of 
message  traffic,  and  portray  cyber  threats  and  responses  with 
increased  fidelity.  Additionally,  DOT&E  estimates  $59  Million 
over  the  FYDP  is  needed  to  provide  additional  capabilities  for 
realistic  threat  development  and  assessment,  as  well  as  additional 
expertise  and  training  for  the  Red  Teams  employing  cyber 
threats  during  training  and  test  events.  Lastly,  DOT&E  estimates 
additional  funding  of  $46  Million  across  the  FYDP  will  support 
assessments  during  all  appropriate  Combatant  Commander  and 
Service  exercises. 

Explosive  Surrogate  for  Use  in  Live  Fire  Test  and  Evaluation 
(LFT&E) 

Title  10,  U.S.  Code  requires  realistic  survivability  testing  of 
combat  and  tactical  wheeled  vehicles  against  combat-realistic 
threats.  DOT&E  oversees  survivability  testing  and  has  concerns 
about  our  ability  to  test  operationally  significant  scenarios 
involving  underbody  blast  threats,  thereby  assuring  adequate 
LFT&E  of  military  vehicles  now  and  in  the  future.  The 
significant  concerns  include  the  need  to: 


•  Develop  and  rigorously  characterize  the  performance  of  a 
surrogate  for  the  Home-Made  Explosive  (HME)  threats  now 
prevalent  in  Afghanistan  because  of  their  ease  of  fabrication 
and  efficacy,  both  of  which  argue  for  the  use  of  HME  by 
enemies  in  future  conflicts.  Currently,  LFT&E  cannot 

use  HME  surrogates  because  the  performance  of  those 
surrogates  (that  is,  the  blast  impulse  they  deliver  to  a  vehicle) 
vary  greatly  from  test  to  test  for  reasons  that  are  not  fully 
understood.  LFT&E  requires  repeatability  from  test  to  test, 
which  can  be  achieved  only  when  the  key  factors  determining 
the  performance  of  HME  surrogates  are  known  and  can  be 
controlled. 

•  More  fully  characterize  the  performance  of  buried  TNT 
devices,  which  are  the  standard  threats  currently  used  in 
LFT&E.  The  blast  impulse  delivered  by  buried  TNT  to 
a  vehicle  is  repeatable  under  the  relatively  narrow  set  of 
conditions  now  used  in  LFT&E.  However,  there  is  a  need  to 
test  and  compare  results  obtained  under  more  widely  varying 
conditions. 

•  Characterize  how  differences  in  soil  composition  affect  the 
blast  impulse  delivered  to  vehicles  undergoing  underbody 
blast  testing.  The  currently  used  single  standard  soil  presents 
multiple  limitations  to  testing  for  numerous  reasons.  It  does 
not  allow  for  roadbed  compaction,  realistic  under-wheel 
threat  placements,  varying  moisture  content,  or  varying 

soil  composition.  This  constrains  the  LFT&E  community’s 
ability  to  evaluate  vehicle  vulnerability  to  underbody  blast 
threats  for  certain  critical,  operationally  significant  scenarios. 
Additionally,  the  Army  may  soon  be  unable  to  continue 
procuring  soils  that  meet  the  current  standard. 

Proposed  funding  of  $15  Million  for  FY13-15  will  enable  the 
Army  to  complete  the  full  scope  of  effort  required  to  ensure 
adequate  LFT&E  of  military  vehicles  against  underbody  blast 
threats.  Understanding  of  blast  threats  will  enable  improvements 
to  vehicle  design  and  capabilities  against  underbody  blast  threats 
that  will  save  lives  in  current  and  future  conflicts. 

Tactical  Engagement  Simulation  (TES)  and  Real  Time  Casualty 
Assessment  (RTCA) 

New  and  upgraded  combat  systems  cannot  be  adequately 
evaluated  without  the  exchange  of  simulated  fire  between  friendly 
and  opposing  forces  in  an  operationally  realistic  environment. 
These  force-on-force  battles  must  contain  enough  realism  to 
cause  Soldiers  and  their  units  to  make  tactical  decisions  and  react 
to  the  real-time  conditions  on  the  battlefield.  Real  Time  Casualty 
Assessment  (RTCA)  is  needed  to  ensure  that  the  simulated 
engagements  have  realistic  outcomes  based  on  the  lethality 
and  survivability  characteristics  of  both  the  systems  under  test 
and  the  opposing  threat  systems.  Future  Tactical  Engagement 
Simulation  (TES)/RTCA  systems  must  include  critical  attributes 
of  real-world  combat  engagements  such  as  direct  and  indirect 
fires,  IEDs  and  mines,  realistic  battle  damage  and  casualties,  a 
mix  of  ground  and  air  vehicles,  and  a  competent  and  capable 
threat  force.  TES/RTCA  systems  must  be  able  to  record  the 
time,  space,  position;  and  firing,  damage,  and  casualty  data  for  all 
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players  in  the  test  event.  Current  TES  systems  cannot  support  the 
simulated  engagements  or  the  data  collection  needed  for  OT&E. 

Past  efforts  by  the  Army  test  community  to  develop  a  TES/RTCA 
capability  have  failed  due  to  affordability  issues.  Army 
Operational  Test  Command  has  begun  a  study  to  review  the 
operational  test  requirements  for  RTCA,  examine  applicable 
technologies,  and  recommend  a  course  of  action  for  future 
developments.  The  Army  has  also  initiated  work  on  the  Army 
Tactical  Engagement  Simulation  System  (A-TESS)  program, 
which  seeks  to  increase  interoperability  and  provide  interface 
standards  for  future  capability  growth  of  its  TES  systems. 

The  offices  responsible  for  test  and  training  in  the  Army  are 
committed  to  working  together  toward  a  future  system  due 
to  their  common  requirements  and  limited  budgets.  DOT&E 
supports  these  initiatives  and  will  continue  to  require  an  adequate 
RTCA  system  to  support  the  OT&E  of  combat  systems  such 
as  the  Ground  Combat  Vehicle,  Abrams  Tank  Modernization, 
Armored  Multipurpose  Vehicle,  Apache  Block  III,  Joint 
Lightweight  Tactical  Vehicle,  and  Stryker  upgrades.  The 
estimated  cost  to  develop  this  capability  is  $125  Million. 

Joint  Urban  Test  Capability  (JUTC) 

Operations  in  urban  environments  present  unique  challenges 
to  Service  members  and  their  equipment.  Degraded  mobility, 
communications,  and  situational  awareness;  a  large  civilian 
presence;  the  risk  of  collateral  damage;  reduced  stand-off 
distances;  and  unique  threat  profiles  are  some  of  the  obstacles 
present  during  urban  operations.  These  challenges  justify  the 
requirement  that  systems  be  tested  in  operationally  realistic  urban 
environments.  The  Army-led  Urban  Environment  Test  Capability 
(UETC)  study  has  determined  the  resources  needed  to  support  the 
T&E  of  systems  operating  in  urban  environments.  The  UETC 
summarized  this  information,  along  with  a  study  of  global  urban 
landscapes,  in  its  final  report.  The  UETC  report  is  the  foundation 
for  the  Joint  Urban  Test  Capability  (JUTC)  project  being  led 
by  the  Army  and  funded  through  the  OSD  Central  Test  and 
Evaluation  Investment  Program. 

DOT&E  is  supportive  of  the  JUTC  requirement;  however,  the 
proposed  physical  surface  urban  area  of  200  meters  by  240 
meters  will  not  be  large  enough  to  support  operational  testing  of 
Company  size  and  greater  test  scenarios.  DOT&E  recommends 
that  the  urban  area  be  expanded  to  the  JUTC  objective 
requirement  of  900  meters  by  900  meters  to  support  future 
operational  test  events.  The  cost  of  the  current  JUTC  effort  is 
estimated  at  $75-95  Million.  DOT&E  is  coordinating  with  the 
TRMC  on  the  feasibility  of  expanding  the  JUTC  to  the  larger 
area. 

Fifth-Generation  Aerial  Target 

No  U.S.  aerial  target  (including  the  QF-16  currently  in 
development)  can  replicate  fifth-generation  fighter  characteristics 
such  as  low  observability  or  embedded  electronic  attack.  As 
a  result,  operationally  realistic  testing  cannot  be  accomplished 
for  U.S.  air-to-air  and  surface-to-air  weapons  systems  against 
fifth-generation  fighters.  Therefore,  DOT&E  is  executing  a 
target  design  and  cost  study  based  on  the  recommendation  of 


the  Defense  Science  Board  with  a  goal  of  determining  if  an 
affordable  Fifth-Generation  Aerial  Target  can  be  developed. 

The  contractor  and  Institute  for  Defense  Analysis  (IDA) 
provided  preliminary  cost  estimates  in  FY12.  DOT&E  will 
focus  on  mitigating  risk  and  resolving  cost  uncertainty  as  the 
study  continues  in  FY13.  DOT&E  is  seeking  $60  Million  to 
develop  a  production-representative  prototype(s)  to  validate 
cost  and  performance  during  flight  test  in  FY15.  This  capability 
is  required  for  adequate  developmental  and  operational  testing 
of  the  Department’s  ongoing  and  planned  aircraft  and  missile 
programs. 

Automated  Test  Capabilities  for  Software-Based  Systems 

Current  acquisition  policy  (DoDI  5000.02)  requires 
“manufacturing  processes  under  control”  as  an  entrance  criterion 
for  a  Milestone  C  decision  for  full-rate  production.  DOT&E 
now  requires  an  equivalent  concept  for  software  acquisitions 
and  software  components  of  a  weapon  system  at  IOT&E. 
Specifically,  program  managers  must  demonstrate  software 
system  sustainment  maturity,  including  program-conducted  T&E 
in  support  of  routine  technology  upgrades.  A  demonstration  of 
sustainment  maturity  will  include  a  demonstration  of  applicable 
test  automation  and  the  ability  to  perform  an  end-to-end  trace  of 
test  information  from  requirements  to  test  scripts  and  defects. 

We  anticipate  that  most  programs  will  take  several  years  to 
create  a  software  test  automation  approach  that  will  satisfy 
the  DOT&E  requirements.  Currently,  programs  complete  test 
automation  and  an  end-to-end  trace  of  test  information  on  a 
per  program  basis  at  varying  levels  of  adequacy.  Very  few 
acquisition  programs  have  mature  test  automation  solutions  for 
regression  testing  that  can  be  demonstrated  at  IOT&E,  and  even 
fewer  programs  can  create  the  environments  and  conditions  to 
validate  their  regression  testing  processes.  Without  substantial 
help  from  a  central  resource,  it  is  likely  that  most  programs  will 
have  this  deficiency  during  IOT&E. 

The  need  for  software  test  automation  strategies  to  satisfy 
DOT&E  requirements  will  create  demand  for  test  automation 
expertise  in  program  offices.  Program  managers  need  a 
resource,  such  as  a  center  of  excellence,  to  help  meet  that 
demand  and  mitigate  problems.  A  center  of  excellence  will 
work  with  vendors  and  government  providers  to  promote 
interoperability  of  Test  as  a  Service  (TaaS)  and  other  test 
automation  solutions.  This  center  of  excellence  is  intended  to: 

•  Centralize  knowledge  of  the  various  automation  approaches 

•  Assist  programs  in  applying  software  test  automation 

•  Create  “in-house”  software  test  automation  expertise 

A  center  of  excellence  may  lessen  the  tendency  to  use  a 
“stove -piped”  approach  to  testing,  may  reduce  duplicative 
resources  (technological  and  human),  should  increase  programs’ 
use  of  existing  capabilities,  and  should  improve  the  consistency 
and  adequacy  in  the  types  of  testing  accomplished.  DOT&E 
estimates  the  DoD  will  need  $2  Million  over  the  next  two  years 
to  establish  a  software  test  automation  center  of  excellence 
that  will  subsequently  be  self-supported  through  fees  for  test 
services. 
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Steerable  Antenna  for  GQM-163A  Supersonic  Target  Testing 

A  steerable  antenna  unit  is  required  to  provide  operationally 
realistic  emissions  from  the  GQM-163A  supersonic  target  that 
will  stay  locked  on  the  target  ship.  This  unit  is  needed  to  ensure 
the  shipboard  EW  system  has  constant  track  of  the  incoming 
target  emissions  so  that  Rolling  Airframe  Missile  Block  1  and / or 
Block  2  missiles  can  then  be  launched  (and  guide  on  those 
same  emissions)  as  interceptors.  This  unit  would  be  similar  to 
the  STEERAN  unit  currently  used  in  the  BQM-74E  subsonic 
target.  The  diameters  of  the  GQM  and  BQM  targets  differ 
greatly,  so  extensive  re-engineering  and  testing  will  be  needed 
to  adapt  the  BQM  unit  to  fit  the  GQM  without  disturbing  the 
GQM  kinematics/maneuverability.  This  capability  is  required 
for  adequate  operational  testing  of  the  CVN-78/Rolling  Airframe 
Missile  Block  2  in  FY17.  Estimated  development  cost  is 
$10-20  Million.  Estimated  unit  cost  is  $500  thousand. 

Additional  EW  Simulator  Units  for  Surface  EW  Improvement 
Program  (SEWIP)  Block  2  OT 

At  present,  there  exists  only  one  each  of  the  Kappa,  Uniform,  and 
Gamma  EW  simulators.  These  simulators  are  flown  on  Lear  Jets 
against  shipboard  EW  systems.  SEWIP  Block  2  is  the  latest  EW 
system  under  development.  Two  of  these  simulators  are  needed 
(one  for  each  Lear  Jet)  so  that  threat-realistic  stream  raid  profiles 
can  be  used  to  adequately  test  the  SEWIP  Block  2  in  FY14.  An 
estimated  development/procurement  cost  is  $5  Million. 

Continuing  Radio  Frequency  Spectrum  Concerns 

The  T&E  community  competes  with  commercial  and  other 
federal  entities  for  access  to  the  RF  spectrum.  RF  spectrum 
allocated  to  commercial  uses  has  increased  due  to  reallocation 
of  the  government  spectrum  and  petitioning  of  the  Federal 
Communications  Commission  (FCC)  for  additional  frequency 
assignments.  The  result  is  insufficient  spectrum  to  support 
T&E  telemetry  operations  (primarily  in  the  L  and  S  frequency 
bands)  and  FCC  restrictions  on  DoD  RF  emissions  and  jamming 
operations  (so  as  not  to  interfere  with  commercial  RF  use).  This 
problem  is  exacerbated  by  the  growth  in  data  transmission  rates 
needed  as  more  complex  weapon  systems  are  developed  (by  the 
U.S.,  allies,  and  adversaries)  and  as  the  military  must  demonstrate 
RF  spectrum  exploitation  to  disrupt  and  deny  spectrum  access  by 
adversaries.  Funding  and  support  from  Congress  as  well  as  other 
federal  agencies  is  needed  to  ensure  adequate  RF  spectrum  to 
support  T&E.  The  objective  would  be  to  pursue  the  following: 

•  Protect  critical  T&E  RF  spectrum  bands  from  reallocation. 

•  Acquire  additional  RF  spectrum  to  offset  reallocated 
spectrum.  This  would  include  development  of  a  multi-Service 
implementation  plan  to  ensure  acquisition  programs  and 
range  facilities  implement  the  means  to  utilize  the  additional 
spectrum. 

•  Develop  methods  and  technologies  that  more  efficiently  use 
the  RF  spectrum. 

TRMC  estimates  the  cost  to  retain  the  current  capacity  of  the 
ranges  (i.e.,  the  number  of  test  operations)  is  on  the  order  of 


$400  Million  over  five  years  due  to  continued  growth  of  data 
transmission  rates,  the  associated  costs  of  developing  the 
technologies  needed  to  support  these  data  transmission  rates, 
and  continuing  encroachment  on  the  spectrum  needed  for 
testing. 

Renewable  Energy  Infrastructure  Impact  on  Operational 
Testing 

Testing  and  evaluation  of  weapons,  sensors,  command  and 
control  networks,  and  other  sensitive  technologies  often  require 
an  electromagnetic  environment  free  of  interference.  The 
proliferation  of  wind  and  solar  renewable  energy  projects  has 
had  an  increasing  impact  on  DoD  testing  and  evaluation  in 
an  electromagnetic  environment.  Although  most  renewable 
energy  projects  are  compatible  with  the  DoD  test  capabilities, 
in  some  cases,  they  can  interfere  with  test  range  instrumentation 
resources  and  systems  under  test.  Collocation  of  wind  and 
solar  power  renewable  energy  projects  with  test  resources  is 
requiring  careful  evaluation  and  investigation  of  a  variety  of 
potential  mitigation  strategies,  since  many  of  the  Nation’s 
most  productive  wind  and  solar  energy  resources  exist  in  close 
proximity  to  some  of  the  DoD’s  most  critical  test  ranges.  For 
example,  wind  farms  located  in  the  Tehachapi  Mountains 
in  California  currently  restrict  the  ability  to  test  certain 
airborne  radar  systems  along  west-to-east  approach  vectors 
to  simulated  targets.  Additionally,  proposed  new  renewable 
energy  developments  in  the  northern  Mojave  Desert  could  more 
severely  affect  these  tests  on  the  east-to-west  approach  vector. 
These  renewable  energy  resources  may  eliminate  the  ability  of 
the  DoD  to  validate  design  parameters  of  radar  systems  in  the 
southern  California/Nevada  region  test  ranges. 

There  are  known  sources  of  interference  with  test  range 
capabilities  from  energy  infrastructure  projects,  such  as  those 
identified  in  California  and  Nevada.  Since  DoD  is  only 
beginning  to  evaluate  such  areas  of  interference  because 
renewable  energy  development  on  a  large  scale  did  not  occur 
until  recently,  more  data  on  interference  from  the  various 
types  of  renewable  energy  projects  are  needed  to  determine 
the  significance  of  its  effects.  Similarly,  research  is  required  to 
develop  interference  mitigation  techniques  and  technologies. 

DoD  has  observed  interference  in  the  following  areas:  wind 
turbines  on  DoD  radar  systems,  central  solar  power  tower 
impact  on  radar  cross  section  evaluation,  electromagnetic 
interference  from  transmission  lines,  and  physical  obstruction 
from  transmission  lines. 

To  address  these  interference  problems,  DOT&E  is  working 
with  the  DoD  Siting  Clearinghouse  to  evaluate  proposed 
renewable  energy  and  infrastructure  projects  as  well  as  develop 
expertise  in  understanding  interference  issues  and  establish  a 
strategy  for  research  that  will  produce  mitigation  techniques  and 
technologies  to  resolve  interference  issues. 
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Joint  Test  and  Evaluation  (JT&E) 


The  primary  objective  of  the  Joint  Test  and  Evaluation  (JT&E) 
Program  is  to  provide  rapid  solutions  to  operational  deficiencies 
identified  by  the  joint  military  community.  The  program  achieves 
this  objective  by  developing  new  tactics,  techniques,  and 
procedures  (TTPs)  and  rigorously  measuring  the  extent  to  which 
their  use  improves  operational  outcomes.  JT&E  projects  may 
develop  products  that  have  implications  beyond  TTPs.  Sponsoring 
organizations  submit  these  products  to  the  appropriate  Service  or 
Combatant  Command  as  doctrine  change  requests.  Products  from 
JT&E  projects  have  been  incorporated  into  joint  and  multi-Service 
documents  through  the  Joint  Requirements  Oversight  Council 
process  and  through  coordination  with  the  Air,  Land,  Sea 
Application  Center.  The  JT&E  Program  also  develops  operational 
testing  methods  that  have  joint  application.  The  program  is 
complementary  to,  but  not  part  of,  the  acquisition  process. 

The  JT&E  Program  has  re-engineered  its  business  model  in 
response  to  a  budget  reduction  starting  in  FY13.  The  major 
change  is  that  project  life  cycle  is  reduced  to  two  years  from 
the  current  three  years.  To  accommodate  this  shorter  timeline, 
nominating  organizations  must  submit  a  robust  initial  nomination 
package  that  provides  substantial  detail  on  how  they  will  achieve 
test  objectives.  The  new  business  model  of  conducting  shorter  test 
projects  necessitates  establishing  permanent  test  units,  managed 
by  operational  test  agencies,  supporting  test  projects  led  by  the 
operational  sponsor.  This  new  business  model  reduces  the  number 
of  staff  members  needed  to  support  a  joint  test  and  improves  the 
stability  of  personnel  assignments. 

The  program  managed  six  joint  tests  in  FY12  that  focused  on  the 
needs  of  operational  forces.  Projects  annotated  with  an  asterisk  (*) 
closed  in  FY12: 

•  Joint  Advanced  Capability  Employment  (J-ACE) 

•  Joint  Cyber  Operations  (JCO) 

•  Joint  Deployable  Integrated  Air  and  Missile  Defense  (JDIAMD) 

•  Joint  Integration  of  Maritime  Domain  Awareness  for  Homeland 
Defense  (JIMDA)* 


•  Joint  Jamming  Assessment  and  Mitigation  (JJAM)* 

•  Joint  Unmanned  Aircraft  Systems  (UAS)  Digital  Information 
Exchange  (JUDIE) 

The  JT&E  Program  instituted  a  quick  reaction  test  (QRT) 
capability  in  2003  to  respond  to  the  pressing  needs  of  today’s 
deployed  forces.  QRTs  are  less  than  a  year  in  duration  and  solve 
urgent  issues.  The  program  managed  16  QRTs  in  FY12: 

•  Afghanistan  Mission  Network  Coalition  Battlespace 
Management  (AMN-CBM)* 

•  Airborne  Maritime  Moving  Target  Indicator  (AMMTI)* 

•  Battlefield  Airborne  Communications  Node  (BACN) 
Intra-Flight  Data  Link  Subsystem  and  Multi-Domain 
Integration  (BIS-MDI) 

•  Civil  Intelligence  Fusion  Concept  of  Operations  (CIFC) 

•  Computer  Network  Defense  Service  Provider  (CNDSP) 

•  Joint  All-Domain  Situational  Awareness  (J-ADSA) 

•  Joint  Analytical  Network  Assessment  (JANA)* 

•  Joint  Battlespace  Awareness  via  Data  Link  (J-BADL) 

•  Joint  Beyond  Line-of-Sight  Command  and  Control  (JBC2) 

•  Joint  Exploitation  of  Modern  Surface-to-Air  Missile  Systems 
(JEMS)* 

•  Joint  Military  Working  Dog  (JMWD)* 

•  Joint  Passive  Electronic  Radio  Frequency  Emission 
Classification  and  Tracking  II  (J-PERFECT  II)* 

•  Joint  Threat  Assessment  and  Negation  for  Installation 
Infrastructure  Control  Systems  (JTANIICS) 

•  Joint  Vehicle  Protection  and  Survivability  System  (JVPSS)* 

•  Rapid  Development  and  Sustainment  of  Enterprise  Mission 
Services  (RDEMS)* 

•  Unmanned  Aircraft  Systems  -  Airspace  Integration  (UAS-AI) 

The  program  executes  special  projects,  as  directed  by  DOT&E, 
which  address  problems  DoD-wide.  The  program  managed 
one  special  project  in  FY12,  the  Rapid  Acquisition  by  Sniper  IK 
Track  and  Attack  (RASTA). 
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JOINT  ADVANCED  CAPABILITY  EMPLOYMENT  (J-ACE) 

Sponsor/Start  Date:  U.S.  Strategic  Command/ August  2011 

Purpose:  To  develop,  test,  and  evaluate  a  standardized  process  to 
support  the  Joint  Force  Commander’s  ability  to  employ  enhanced, 
advanced  capabilities  to  overcome  complex  targeting  challenges. 

Products/Benefits:  The  J-ACE-developed  products  are: 

•  A  repeatable  operational  employment  process  that  will  enhance 
planning  by  developing,  evaluating,  and  coordinating  concepts 
of  employment  (CONEMPs)  that  can  be  used  by  the  Joint 


Staff,  Combatant  Commands,  Services,  and  National  Security 
Agency  to  solve  complex  targeting  challenges 

•  Multiple  approved,  enhanced,  and  advanced  capability 
CONEMPs  to  overcome  complex  targeting  challenges 

•  Relevant  training  scenarios  and  vignettes 

•  Documented  effects  associated  with  techniques  against 
representative  targets 

•  Recommendations  to  operational  concept  of  operations 
(CONOPS)  and  approval  process  packages 


JT&E  319 


JT&E  PROGRAM 


JOINT  CYBER  OPERATIONS  (JCO) 

Sponsor/Start  Date:  U.S.  Pacific  Command 
(USPACOM)/ August  2010 

Purpose:  To  develop,  assess,  and  evaluate  joint  TTPs  to  employ 
an  adaptive  cyber  defense  Virtual  Secure  Enclave  (VSE)  to  ensure 
the  protection  and  availability  of  critical  command  and  control 
services. 

Produets/Benefits: 

•  Addresses  network  vulnerabilities  of  critical  command  and 
control  services  by  enabling  Joint  Task  Force  Commanders 
to  employ  an  adaptive  cyber  defense  VSE  to  protect  against, 
detect,  and  respond  to  cyber  threats  against  specific  command 
and  control  applications  at  the  operational  level 

•  Provides  the  Commander  with  situational  awareness  and  cyber 
defense  options  to  maintain  a  proactive  defensive  posture 

•  Facilitates  a  systematic  approach  to  warfighting  in  the  cyber 
domain 

•  Tests  and  validates  operational  effectiveness  of  Joint  Task  Force 
implementation 

JOINT  DEPLOYABLE  INTEGRATED  AIR  AND  MISSILE  DEFENSE 
(JDIAMD) 

Sponsor/Start  Date:  North  American  Aerospace  Defense 
(NORAD),  U.S.  Northern  Command  (USNORTHCOM),  Army 
Space  and  Missile  Defense  Command/August  2011 

Purpose:  To  develop  joint  planning  and  execution  processes 
and  procedures  for  deployable,  integrated  air  and  missile  defense 
(IAMD)  for  the  homeland. 

Products/Benefits: 

•  IAMD  process  modeling  that  provides  a  comprehensive  view  of 
the  integrated  planning  and  execution  process 

•  NORAD  and  USNORTHCOM  current  operations  planning 
processes,  checklists,  and  procedures  for  IAMD 

•  Continental  NORAD  Region  and  Air  Forces  North  planning  and 
execution  TTPs  for  IAMD 

•  U.S.  Fleet  Forces  and  Third  Fleet  planning  and  execution  TTPs 
for  naval  support  of  IAMD 

•  Army  North  planning  and  execution  TTPs  for  operational 
control  of  ground-based  IAMD  forces 

•  263rd  Army  Air  and  Missile  Defense  Command  planning  and 
execution  TTPs  for  IAMD 

JOINT  INTEGRATION  OF  MARITIME  DOMAIN  AWARENESS  FOR 
HOMELAND  DEFENSE  (JIMDA)* 

(Closed  October  2012) 

Sponsor/Start  Date:  NORAD  and  USNORTHCOM/August  2009 

Purpose:  To  develop  TTPs  for  consistent  and  comprehensive 
coordination  across  the  distributed  maritime  community  in  order  to 
synchronize  maritime  domain  information  for  key  decision  makers 
across  the  operations  centers  responsible  for  homeland  defense. 

Products/Benefits: 

•  Integrated  maritime  domain  awareness  processes,  procedures, 
and  checklists  for  all  participating  operations  centers 

320  JT&E 


•  An  online  handbook  detailing  the  information  and  resources 
available  for  developing  and  sharing  maritime  domain 
information  across  the  NORAD  and  USNORTHCOM 
command  and  control  network  and  among  participating 
interagency  partners 

•  An  online  portal,  hosted  by  NORAD  and  USNORTHCOM, 
that  enables  the  distributed  execution  of  the  maritime  domain 
awareness  processes,  procedures,  and  checklists  during  daily 
operations  at  participating  operations  centers  throughout  the 
maritime  community 

JOINT  JAMMING  ASSESSMENT  AND  MITIGATION  (JJAM)* 

(Closed  September  2012) 

Sponsor/Start  Date:  Air  Force/August  2009 

Purpose:  To  develop  TTPs  that  mitigate  the  effects  of  adversary 
purposeful  interference  (PI)  on  satellite  communications 
(SATCOM)  and  allow  friendly  operational  forces  to  effectively 
sustain  operations  when  SATCOM  is  degraded  or  denied. 

Products/Benefits: 

•  Created  SATCOM  PI  mitigation  training  employing  live 
jamming  signals  that  allowed  the  SATCOM  community  of 
interest  to  rehearse  technical  and  procedural  PI  resolution 
processes 

•  Established  SATCOM  PI  mitigation  TTPs  and  CONOPS 
being  used  by  U.S.  Strategic  Command,  USPACOM,  and  U.S. 
Central  Command  (USCENTCOM) 

•  Improved  information  exchange,  situational  awareness,  and 
command  and  control  decision  processes  which  reduced  the 
timeline  for  detection,  attribution,  and  target  nomination  from 
days  to  hours 

•  Integrated  Joint  Space  Operations  Center  space  target 
coordination  and  nomination  processes  into  theater  operations 

•  Produced  a  first-generation  PI  visualization  tool  to  provide 
key  decision  makers  real-time  awareness  of  events  affecting 
operationally  critical  SATCOM  networks 

•  Published  SATCOM  PI  Training  and  Exercise  Handbook  for 
Combatant  Command  planners 

•  Integrated  SATCOM  PI  Mitigation  TTPs  into  19  joint  and 
Service  publications 

JOINT  UNMANNED  AIRCRAFT  SYSTEMS  DIGITAL 
INFORMATION  EXCHANGE  (JUDIE) 

Sponsor/Start  Date:  Air  Force/August  2010 

Purpose:  To  develop,  test,  and  evaluate  cross-component  UAS 
information  exchange  TTPs  used  to  improve  joint  battlespace 
situational  awareness  and  target  prosecution  capabilities  for  the 
Commanders  at  the  tactical  level  of  brigade  and  below. 

Products/Benefits: 

•  Standardize  UAS  information  exchange  TTPs  and  checklists 

•  Standardize  terminology  for  UAS  information  exchange 


*  Project  closed  in  FY12 


JT&E  PROGRAM 


•  Integrate  information  portal  and  situational  awareness  display  •  Introduce  the  information  exchange  TTPs  to  combat  training 
technology  currently  in  use  by  the  components  to  improve  the  centers  and  formal  training  units 
UAS  information  exchange  efficiency 
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AFGHANISTAN  MISSION  NETWORK  COALITION 
BATTLESPACE  MANAGEMENT  (AMN-CBM)* 

(Closed  January  2012) 

Sponsor/Start  Date:  USCENTCOM  and  Defense  Information 
Systems  Agency  (DISA)/January  20 1 1 

Purpose:  To  measure  and  evaluate  the  utility  of  battlespace 
management  TTPs  for  both  joint  and  coalition  forces. 

Products/Benefits:  Developed  a  methodology  for  use  by  the 
Afghanistan  Mission  Network  testing  organizations  enabling  them 
to  test  and  evaluate  the  utility  of  TTPs  for  executing  coalition 
mission  threads,  and  to  execute  repeatable  test  events  providing 
reliable  quantitative  data. 

AIRBORNE  MARITIME  MOVING  TARGET  INDICATOR  (AMMTI)* 

(Closed  September  2012) 

Sponsor/Start  Date:  U.S.  Southern  Command/July  2011 

Purpose:  To  develop  TTPs  for  the  employment  of  the 
Joint  Surveillance  Target  Attack  Radar  System  (JSTARS) 
Enhanced  Land  Maritime  Mode  (ELMM)  in  support  of  the 
counter-narcotics  trafficking  mission  in  U.S.  Southern  Command’s 
area  of  responsibility.  AMMTI  will  formidate  the  first  TTPs 
for  employment  of  the  JSTARS  radar  system  for  open  water 
surveillance. 

Products/Benefits: 

•  Optimized  system  parameters  for  JSTARS  maritime 
employment 

•  Detailed  information  on  the  probability  of  detection  and  target 
location  error  for  the  specified  AMMTI  targets 

•  Mission  planning  improvements  for  JSTARS  maritime 
employment 

•  TTPs  for  employment  of  the  JSTARS  ELMM  capability  in 
multiple  Combatant  Commands 

•  Flash  Bulletin  on  JSTARS  maritime  employment 
considerations 

BATTLEFIELD  AIRBORNE  COMMUNICATIONS  NODE  (BACN) 
INTRA-FLIGHT  DATALINK  SUBSYSTEM  AND  MULTI-DOMAIN 
INTEGRATION  (BIS-MDI) 

Sponsor/Start  Date:  USPACOM/November  2011 

Purpose:  To  develop  and  evaluate  TTPs  for  the  BIS-MDI, 
an  upgrade  to  the  basic  BACN  capability  that  provides 
interoperability  across  multi-band  voice  and  datalink 
communications  and  bridges  widely  separated  Link  1 6  networks. 
This  will  greatly  enhance  situational  awareness  and  operational 


effectiveness,  especially  between  fourth  and  fifth  generation 
fighter  aircraft  and  surface  shooters. 

Products/Benefits: 

•  BIS-MDI  fuses  sensor  information  from  multiple  sources, 
including  fourth  and  fifth  generation  platfonns,  into  the 
Service  member’s  overall  common  operational  picture 

•  Joint  and  coalition  Service  member  CONOPS  and  TTPs  for 
BIS-MDI  employment  in  support  of  potential  USPACOM 
operations  conducted  in  anti-access  and  area-denial 
environments 

CIVIL  INTELLIGENCE  FUSION  CONCEPT  OF  OPERATIONS 
(CIFC) 

Sponsor/Start  Date:  Joint  Staff/January  2012 

Purpose:  To  test  and  validate  the  Civil  Intelligence  Fusion 
CONOPS.  Many  intelligence  organizations  do  not  dedicate 
adequate  support  to  collecting  and  integrating  civil  information, 
preventing  the  Joint  Force  Commander  from  obtaining  a  holistic 
view  of  the  operational  environment. 

Produets/Benefits: 

•  Validated  and  improved  CONOPS  for  fusion  of  civil 
intelligence 

•  New  doctrine  change  requests  for  consideration 

•  Connects  sources  of  civil  information  with  planners,  operators, 
and  intelligence  professionals  creating  a  community  of  interest 

•  Provides  processes  and  architecture  for  improved  information 
sharing  resulting  in  better  knowledge  of  the  operational 
environment 

COMPUTER  NETWORK  DEFENSE  SERVICE  PROVIDER 
(CNDSP) 

Sponsor/Start  Date:  DoD  Chief  Information  Office  (CIO)/July 
2012 

Purpose:  To  develop,  document,  and  formalize 
Department-level  TTPs  to  ensure  the  capability  exists  within 
DoD  CNDSPs  to  guide  day-to-day  operations  and  ensure  an 
acceptable  level  of  performance  of  DoD  CNDSP  defenders  when 
facing  a  capable  cyber  adversary. 

Products/Benefits:  The  CNDSP  QRT  will  develop  a 
methodical,  repeatable,  and  verifiable  framework  for  measuring 
the  effectiveness  of  DoD  CNDSPs  on  the  Non-secure  Internet 
Protocol  Router  Network  and  Secret  Internet  Protocol  Router 
Network. 


*  Project  closed  in  FY12 
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JOINT  ALL  DOMAIN  SITUATIONAL  AWARENESS  (J-ADSA) 

Sponsor/Start  Date:  NORAD  and  USNORTHCOM/June  2012 

Purpose:  To  develop  and  test  necessary  TTPs  to  improve  the 
ability  of  NORAD  and  USNORTHCOM  senior  leaders  to  gain 
and  maintain  comprehensive,  integrated  all-domain  situational 
awareness  and  decision  superiority. 

Products/Benefits:  The  J-ADSA  QRT  will  develop  and  evaluate 
the  effectiveness  of  TTPs  to  access  and  analyze  the  effectiveness 
of  seemingly  unrelated  cross-domain  activities  and  events  to 
enable  the  NORAD-USNORTHCOM  leadership  to  make  timely 
decisions. 

JOINT  ANALYTICAL  NETWORK  ASSESSMENT  (JANA)* 

(Closed  January  2012) 

Sponsor/Start  Date:  USNORTHCOM/January  2011 

Purpose:  To  develop,  test,  and  validate  a  mission  essential 
circuit  list  (MECL)  for  all  inter-landmass  command  circuit 
service  designators  in  the  Pacific  theater,  supporting  operations 
plans  and  critical  mission  operations,  both  inside  and  outside  the 
USPACOM  area  of  responsibility. 

Products/Benefits: 

•  Created  a  methodology  utilizing  an  analytical  hierarchy 
process  to  develop  standardized  inter-landmass  MECLs  and 
TTPs  for  use  of  MECLs  during  strategic  communication 
restoration 

•  Command  leadership  can  now  properly  sequence  restoration 
efforts  based  on  inter-landmass  MECLs  to  ensure  the  most 
important  capabilities  are  restored  first 

JOINT  BATTLESPACE  AWARENESS  VIA  DATALINK  (J-BADL) 

Sponsor/Start  Date:  NORAD  and  USNORTHCOM/August 
2012 

Purpose:  To  research  and  develop  TTPs  that  will  focus 
NORAD,  USNORTHCOM,  and  supporting  commands’  use 
of  joint  global  sensor  information  to  provide  cuing  to  address 
priorities,  adjust  surveillance  assets,  or  position  existing  forces  in 
executing  the  Joint  Engagement  Sequence  against  advanced  air 
threats. 

Products/Benefits:  The  expected  test  product  includes  TTPs  that 
lay  out  the  processes  and  procedures  for  the  execution  of  Joint 
Engagement  Sequence  capabilities  that  can  be  used  operationally 
by  NORAD  and  USNORTHCOM,  as  well  as  by  other  Combatant 
Commands  and  government  agencies,  against  advanced  air 
threats. 

JOINT  BEYOND  LINE-OF-SIGHT  COMMAND  AND  CONTROL 
(JBC2) 

Sponsor/Start  Date:  Air  Force/July  2012 

Purpose:  To  develop  and  evaluate  TTPs  so  operations  centers 
can  better  plan  and  execute  integrated  employment  of  Beyond 
Line-of-Sight  Command  and  Control  system  of  systems 


to  support  real-time,  bi-directional,  full  motion  video,  and 
collaborative  command  and  control  capabilities. 

Produets/Benefits:  The  QRT  will  focus  the  TTP  development 
on  operations  at  both  the  Maritime  Operations  Center  and  the 
Air  and  Space  Operations  Center  in  the  USCENTCOM  area 
of  operations  in  support  of  responsive  fleet  defense  and  strike 
operations. 

JOINT  EXPLOITATION  OF  MODERN  SURFACE-TO-AIR  MISSILE 
SYSTEMS  (JEMS)* 

(Closed  January  2012) 

Sponsor/Start  Date:  Naval  Air  Weapons  Center/January  2011 

Purpose:  To  develop  and  test  TTPs  for  effectively  employing 
countermeasures  to  modern  surface-to-air  missile  systems  for 
improving  aircrew  survivability. 

Produets/Benefits:  The  JEMS  project  provided  updates  to  the 
Joint  Research  Assessment  and  Analysis  Center  for  inclusion  in 
applicable  threat  simulations.  It  also  produced  TTPs  and  training 
outlines  for  incorporation  by  the  Services  into  applicable  aircraft 
tactics  and  threat  training  products. 

JOINT  MILITARY  WORKING  DOG  (JMWD)* 

(Closed  April  2012) 

Sponsor/Start  Date:  USCENTCOM/April  2011 

Purpose:  To  develop,  test,  and  validate  TTPs  to  support 
the  ground  tactical  Commander’s  use  and  support  of 
military  working  dogs  while  performing  counter-IED  and 
counterinsurgency  missions. 

Products/Benefits:  The  JMWD  QRT  developed  a  single-source 
document  for  ground  maneuver  leaders  to  plan  and  execute 
counter-IED  and  counterinsurgency  missions  employing 
military  working  dogs.  The  primary  test  products  included 
the  Integrating  Military  Working  Dogs  into  Counter-IED  and 
Counterinsurgency  Operations  Handbook  and  a  quick  reference 
card.  JT&E  transitioned  both  of  these  to  the  U.S.  Army  Military 
Police  School  with  10,000  copies  printed  for  both  training  and 
operational  use. 

JOINT  PASSIVE  ELECTRONIC  RADIO  FREQUENCY  EMISSION 
CLASSIFICATION  AND  TRACKING  (J-PERFECT)  II  * 

(Closed  July  2012) 

Sponsor/Start  Date:  NORAD  and  USNORTHCOM/May  2011 

Purpose:  To  develop  and  evaluate  CONOPS  and  TTPs  for 
employing  DoD  intelligence,  surveillance,  and  reconnaissance 
capabilities  that  have  the  potential  to  improve  near  real-time 
situational  awareness  for  decision  makers  and  that  will  support 
successful  operations  against  modern  strategic  threats  to  the 
United  States  and  Canada. 
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Products/Benefits: 

•  Detect,  track,  and  identify  strategic  air  threats  to  North 
America 

•  Outline  the  actions  and  methods  which  implement  doctrine 
and  describe  how  forces  will  be  employed  in  operations 

•  Make  recommendations  to  Combatant  Commands  and 
applicable  national  agencies  other  than  NORAD  and 
USNORTHCOM 

JOINTTHREAT  ASSESSMENT  AND  NEGATION  FOR 
INSTALLATION  INFRASTRUCTURE  CONTROL  SYSTEMS 
(JTANIICS) 

Sponsor/Start  Date:  Air  Force/January  2012 

Purpose:  To  develop  and  validate  a  risk  assessment  handbook 
for  use  by  installation  Commanders  to  strengthen  their  industrial 
control  system  security  posture. 

Products/Benefits:  The  JTANIICS  handbook: 

•  Enables  the  installation  Commander  to  conduct 
self-assessments  of  industrial  control  system  vulnerabilities 

•  Provides  guidelines  for  assigning  priority  to  vulnerabilities 
based  on  mission  requirements 

•  Includes  a  validated  methodology  that  aids  in  identifying 
commonly  overlooked  systems  that  can  potentially  allow 
unauthorized  access  to  mission-critical  and  safety-critical 
systems 

JOINT  VEHICLE  PROTECTION  AND  SURVIVABILITY  SYSTEMS 
(JVPSS)* 

(Closed  July  2012) 

Sponsor/Start  Date:  USCENTCOM/July  2011 

Purpose:  To  capture  lessons  learned  and  best  practices  to 
identify,  document,  and  validate  the  safety  and  survivability 
features  of  the  primary  ground  combat  vehicles  used  in  the 
USCENTCOM  area  of  responsibility. 

Products/Benefits:  A  handbook  and  training  video  that 
contains  TTPs,  lessons  learned,  and  best  practices  that  maximize 
Service  member  survivability  when  utilizing  Mine  Resistant 
Ambush  Protected  Vehicles,  Strykers,  and  personnel  protective 


equipment.  The  handbook  targets  the  individual  vehicle  operator, 
crew  members,  and  small  unit  leader  with  instructions  on  how 
to  maximize  the  effectiveness  of  the  safety  and  survivability 
features  of  their  vehicle.  These  instructions  are  listed  as  “do’s” 
and  “don’ts”  in  a  graphically  intensive,  reader-friendly  format. 

RAPID  DEVELOPMENT  AND  SUSTAINMENT  OF  ENTERPRISE 
MISSION  SERVICES  (RDEMS)* 

(Closed  July  2012) 

Sponsor/Start  Date:  DoD  ClO/July  2011 

Purpose:  To  identify  and  document  impediments  to  the 
implementation  of  the  DoD  Information  Sharing  Strategy.  The 
resulting  document  reflects  the  lessons  learned  and  updates  the 
CONOPS  and  TTPs  in  an  RDEMS  User  Guide,  based  upon 
DoD-prescribed  standards,  specifications,  doctrine,  and  technical 
processes  to  enable  effective  and  efficient  enterprise  information 
sharing. 

Products/Benefits: 

•  Documented  impediments  to  information  sharing  at  the  joint 
level  across  Combatant  Commands,  Services,  and  other 
Department  organizations 

•  DoD  CIO  has  taken  the  impediments  for  action  and  is  actively 
working  to  resolve  them 

•  Product  sponsors  captured  lessons  learned  from  the  project 
use  case  and  updated  CONOPS  and  TTPs  in  the  RDEMS  User 
Guide  for  use  throughout  the  Department 

UNMANNED  AIRCRAFT  SYSTEMS  -  AIRSPACE  INTEGRATION 
(UAS-AI) 

Sponsor/Start  Date:  NORAD  and  USNORTFICOM/January 
2012 

Purpose:  To  evaluate  a  discrete  set  of  flight  access  profiles  from 
the  DoD  Airspace  Integration  CONOPS  developed  to  facilitate 
UAS  integration  into  the  National  Airspace  System 

Products/Benefits:  Recommend  potential  improvements  to  the 
CONOPS  and  provide  all  test  results  to  the  USD(AT&L)  UAS 
Task  Force  for  the  purpose  of  identifying  CONOPS  gaps  revealed 
by  the  QRT. 


SPECIAL 

RAPID  ACQUISITION  BY  SNIPER  IK  TRACK  AND  ATTACK 
(RASTA) 

Sponsor/Start  Date:  USPACOM/November  2011 

Purpose:  To  develop  and  test  joint  TTPs  that  improve  the 
timely  integration  of  specific  target  track  generating  capabilities 
for  tactical  fighter  aircraft  during  combat  employment  in 
an  environment  that  includes  Advanced  Electronic  Attack 
waveforms.  Concurrent  TTP  development  lays  out  the  process 
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for  open  architecture  changes  and  additions  to  the  operational 
flight  program  for  fighter  aircraft  to  enable  changes  to  be  made 
rapidly  and  inexpensively. 

Products/Benefits:  RASTA  will  provide  Service  members  the 
ability  to  generate  target-quality  information  to  enhance  kill  chain 
effectiveness  while  operating  in  an  Advanced  Electronic  Attack 
waveform  environment. 


*  Project  closed  in  FY12 
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The  Center  for  Countermeasures  (the  Center)  is  a  joint  activity 
that  directs,  coordinates,  supports,  and  conducts  independent 
countermeasure/counter-countermeasure  (CM/CCM)  T&E 
activities  of  U.S.  and  foreign  weapon  systems,  subsystems, 
sensors,  and  related  components.  The  Center  accomplishes 
this  work  in  support  of  the  DOT&E,  Deputy  Assistant 
Secretary  of  Defense  (DASD)  for  Developmental  Test  and 
Evaluation  (DT&E),  weapon  system  developers,  and  the 
Services.  The  Center’s  testing  and  analyses  directly  support 
evaluation  of  the  operational  effectiveness  and  suitability  of 
CM/CCM  systems. 

Specifically,  the  Center: 

•  Performs  early  assessments  of  CM  effectiveness  against 
threat  and  DoD  systems  and  subsystems. 

•  Determines  performance  and  limitations  of  missile  warning 
and  aircraft  survivability  equipment  (ASE)  used  on 
rotary-wing  and  fixed-wing  aircraft. 

•  Determines  effectiveness  of  precision-guided  weapon 
(PGW)  systems  and  subsystems  when  operating  in  an 
environment  degraded  by  CM. 

•  Develops  and  evaluates  CM/CCM  techniques  and  devices. 

•  Develops  and  tests  new  CMs  in  operationally  realistic 
environments. 

•  Provides  analysis  and  recommendations  on  CM/CCM 
effectiveness  to  Service  Program  Offices,  DOT&E, 
DASD(DT&E),  and  the  Services. 

•  Supports  Service  member  exercises,  training,  and 
pre -deployment  activities. 


During  FY12,  the  Center  tested,  analyzed,  and  reported  on  more 
than  40  DoD  electro-optical  systems  or  subsystems  with  special 
emphasis  on  rotary-wing  survivability.  The  Center  participated 
in  operational/developmental  tests  for  rotary-  and  fixed-wing 
ASE,  PGWs,  hostile  fire  indicator  (HFI)  data  collection, 
experimentation  tests,  and  pre-deployment/exercise  support 
involving  the  use  of  CM/CCM. 

Approximately  53  percent  of  the  Center’s  efforts  were  spent 
on  ASE  and  F1F1  systems,  and  7  percent  of  the  Center’s  efforts 
were  focused  on  overseas  contingency  operations  support  (with 
emphasis  on  CM-based,  pre-deployment  training  for  rotary-wing 
units).  About  19  percent  of  the  Center’s  efforts  were  spent  on 
PGW,  foreign  system,  and  other  types  of  field  testing  not  related 
to  ASE,  and  15  percent  were  applied  to  internal  improvement  and 
modernization  efforts  to  enhance  test  capabilities  and  efforts  to 
develop  test  methodologies  for  use  across  the  Services. 

The  Center  continued  to  develop  multiple  test  tools  for  evaluating 
ASE  infrared  countermeasure  (1RCM)  systems  and  hostile  fire 
signature  (FISIG)  database  models  used  to  support  development 
of  HFI  systems.  In  addition  to  leading  test  tool  development 
efforts,  the  Center  also  developed  an  ASE  T&E  methodology 
guidebook  to  provide  DoD  with  guidance  for  planning,  executing, 
and  reporting  on  ASE  test  events.  The  Center  dedicates  about 
6  percent  of  its  efforts  to  providing  subject  matter  expertise  to 
numerous  working  groups  and  task  forces. 

The  following  activities  are  representative  of  those  conducted  by 
the  Center  during  the  past  year. 


ASEANDHSI  ACTIVITIES 


ROTARY-WING  TEST  EVENTS 

Army:  Reduced  Optical  Signature  Emissions 

Solution  VII 

•  Sponsor:  Department  of  the  Army  Technology 
Applications  Program  Office,  Systems  Integration  and 
Maintenance  Office,  Aircraft  Survivability  Equipment 
Cell 

•  Activity:  The  Center  provided  test  assets  and  crew  to 
perform  effectiveness  testing  of  flares  and  flare  sequences 
against  reactive  captive  infrared  (IR)  missiles.  These  tests 
evaluated  new  CM  sequences,  variations  of  current  CM 
sequences  using  improved  flares,  or  different  flares  within 
the  sequences.  The  Army  used  these  data  to  finalize  flare 
sequences  on  160th  Special  Operations  Aviation  Regiment 
rotary-wing  aircraft. 

•  Benefit:  The  outcome  of  this  combined  effort  resulted  in 
verification  of  the  effectiveness  of  flare  sequences  used  on 
aircraft  deployed  in-theater  and  under  development. 


Army:  MH-60M  Initial  Operational  Capabilities  Testing 
and  Training 

•  Sponsor:  Department  of  the  Army  Technology  Applications 
Program  Office,  Systems  Integration  and  Maintenance 
Office,  Aircraft  Survivability  Equipment  Cell 

•  Activity:  The  Center  provided  Joint  Mobile  IRCM  Test 
System  (JMITS)  simulations,  reactive  captive  IR  seekers, 
and  test  personnel  to  conduct  integrated  testing  and  aircrew 
training  of  the  aircraft  Common  Missile  Warning  Sensor  and 
IR  flare  dispensers.  The  performance  of  the  aircraft’s  IR  flare 
sequence  against  reactive  captive  IR  missiles  was  evaluated 
along  with  aircrew  training  in  an  electronic  warfare  (EW) 
threat  environment  consisting  of  the  JMITS  simulations 

and  radio  frequency  threats.  The  Army  used  these  data  to 
finalize  flare  sequences  on  1 60th  Special  Operations  Aviation 
Regiment  MH-60M  rotary-wing  aircraft  and  develop  tactics, 
techniques,  and  procedures  for  the  aircrews. 
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•  Benefit:  The  outcome  of  this  combined  effort  resulted  in 
verification  of  the  effectiveness  of  flare  sequences  used  on 
MH-60M  aircraft  and  aircrew  training  in  a  simulated  EW 
threat  environment. 

Army:  Hostile  Fire  Indicting  System  (HFIS)  -  Army  Flight 
Test  2 

•  Sponsor:  U.S.  Special  Operations  Command  (USSOCOM), 
Technology  Applications  Program  Office 

•  Activity:  The  Center  provided  a  laser  system  for  laser 
simulations  to  support  a  flight  data  collection  event  with 
USSOCOM  MH-47  and  MH-60  aircraft  equipped  with  the 
HFIS  system  at  Redstone  Arsenal,  Alabama. 

•  Benefit:  The  sponsor  used  this  event  to  collect  background 
and  live  fire  data  from  the  AN/AAR-57  Common  Missile 
Warning  System,  AN/AVR-2B  Laser  Detecting  Set,  and 
Helicopter  Alert  Threat  Termination- Acoustics  systems 
installed  on  representative  aircraft. 

Navy:  Technology  Demonstration  of  the  Department  of 
the  Navy  (DoN)  Large  Aircraft  Infrared  Countermeasures 
(LAIRCM)  Interface  to  the  AN/ALE-47  for  Smart  Dispense 
and  Advanced  IR  Countermeasure  Techniques 

•  Sponsor:  Navy  Program  Executive  Officer,  Advanced 
Tactical  Aircraft  Protection  Systems  Program  Office 
(PMA  272),  and  Naval  Research  Laboratory 

•  Activity:  The  Center  provided  JMITS  IR  stimulations  to 
verify  the  performance  of  advanced  IRCM  techniques  and  a 
new  interface  between  the  DoN  LAIRCM  and  ALE-47.  The 
Center  provided  all  data  collected  to  the  sponsors  for  their 
assessments. 

•  Benefit:  The  data  collected  from  this  effort  allowed  the 
sponsors  to  perform  early  assessments  of  the  new  interface 
between  the  DoN  LAIRCM  and  ALE-47  for  smart  dispense 
techniques.  In  addition,  it  allowed  the  sponsors  to  assess 
the  performance  of  the  advanced  IRCM  techniques  against 
reactive  IR  static  threat  seekers  and  modify  these  advanced 
IRCM  techniques  for  improved  performance. 

Navy:  Distributed  Aperture  IR  Countermeasures  (DAIRCM) 
MH-60R  Test 

•  Sponsor:  Naval  Research  Laboratory 

•  Activity:  The  Center  provided  JMITS  IR  simulations  to 
verify  the  performance  of  the  DAIRCM  jammer  against  static 
seekers.  The  Center  provided  all  JMITS  and  static  seeker  data 
to  the  sponsor  for  assessment. 

•  Benefit:  The  sponsor  used  the  data  from  this  test  effort  to 
adjust  their  jam  code,  which  resulted  in  better  performance 
against  the  static  seekers. 

Navy/Marine  Corps:  AH-1Z  and  MV-22  Aircraft 
Self-Protection  Optimization  Flight  Tests 

•  Sponsor:  Naval  Surface  Warfare  Center  -  Crane  Division, 
with  funding  from  the  Aircraft  Self-Protection  Optimization 
program 

•  Activity:  The  Center  provided  test  assets  and  crew  to  perform 
effectiveness  testing  of  flares  and  flare  sequences  against 
reactive  captive  IR  missiles.  These  tests  evaluated  new  CM 


sequences,  variations  of  current  CM  sequences  using  improved 
flares,  or  different  flares  within  the  sequences. 

•  Benefit:  The  sponsors  are  using  the  test  results  on  flare 
sequence  effectiveness  to  enhance  the  protection  of  these 
aircraft  against  IR  Man-Portable  Air  Defense  Systems 
(MANPADS). 

OSD:  Rotorcraft  Aircraft  Survivability  Equipment  (RASE) 

Experiment  2012 

•  Sponsor:  Assistant  Secretary  of  Defense  for  Research  and 
Engineering 

•  Activity:  The  Center  served  as  Experiment  Director  and 
radiometric  data  collector  during  the  RASE  2012  Tower  event 
at  the  Weapons  Survivability  Laboratory  Remote  Test  Site, 
China  Lake,  California.  Fourteen  different  systems  mounted 
on  an  SH-60  helicopter  installed  on  a  hover  stand  participated 
in  the  experiment. 

•  Benefit:  The  RASE  Experiment  is  a  venue  focused  on 
ASE  that  enhances  decision  makers’  understanding  of  ASE 
performance  and  advances  the  ASE  state-of-the-art  testing. 

The  RASE  Experiment  is  expected  to  improve  realism  and 
standardization  in  the  testing  of  ASE,  improve  the  extent 
of  testing  prior  to  fielding,  and  provide  an  opportunity  for 
multiple  developers  to  save  costs  overall. 

FIXED-WING  TEST  EVENTS 

Air  Force:  LAIRCM  Operational  Flight  Program  (OFP)-14X 

Software  Update 

•  Sponsor:  46th  Test  Wing,  EglinAFB 

•  Activity:  The  Center  provided  JMITS  missile  simulators  and 
crews  to  perform  ultraviolet  and  two-color  IR  simulations 

to  collect  system  response  data  for  assessing  the  OFP-14X 
software  update.  The  Air  Force  conducted  the  tests  at  Eglin 
AFB  using  a  Beechcraft  King  Air  aircraft  fitted  with  a 
LAIRCM  test  bed  capable  of  supporting  ultraviolet  and  IR 
missile  warning  systems  and  the  Viper™  laser. 

•  Benefit:  The  testing  provided  the  Air  Force  with  a 
cost-effective  test  venue  for  collecting  critical  data  needed  to 
assess  performance  of  the  LAIRCM  flight  test  software  prior 
to  installation  on  the  host  aircraft. 

ROTARY-  AND  FIXED-WING  TEST  EVENTS 

Air  Force,  Navy:  Advanced  Strategic  and  Tactical  IR 

Expendables 

•  Sponsors:  Naval  Surface  Warfare  Center  -  Crane  Division, 
Air  Force  Special  Operations  Command,  46th  Test  Wing,  and 
Air  Mobility  Command 

•  Activity:  The  Center  provided  test  assets  and  crew  to  collect 
test  data  on  eight  different  aircraft  against  reactive  captive 
IR  missiles.  These  tests  evaluated  new  flare  CM  sequences, 
variations  of  current  flare  CM  sequences  using  improved 
flares,  or  different  flares  within  the  sequences. 

•  Benefit:  Sponsors  are  using  these  test  results  on  flare 
sequence  effectiveness  to  enhance  the  protection  of  various 
aircraft  against  IR  MANPADS. 
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Army:  Seeker  Bowl  VII 

•  Sponsors:  U.S.  Army  Research  Development  and 
Engineering  Command,  Engineer  Research  and  Development 
Center,  and  Aviation  Applied  Technology 

•  Activity:  The  Center  provided  test  assets  and  crew  to  collect 
test  data  on  flare  protection  effectiveness  for  four  fixed-wing 
and  two  rotary-wing  aircraft  against  reactive  captive  IR 
missiles.  The  test  evaluated  the  effectiveness  of  new  flare 
CM  sequences  or  variations  of  current  flare  CM  sequences. 

•  Benefit:  Sponsors  are  using  these  test  results  on  flare 
sequence  effectiveness  to  enhance  the  protection  of  various 
aircraft  against  IR  MANPADS. 

HOSTILE  FIRE  INDICATOR  (HFI)  DATA  COLLECTION  EVENTS 

Army:  Hostile  Fire  Detection  System  Signature  Ammo 

Study  (SAS) 

•  Sponsor:  Program  Manager  -  Aircraft  Survivability 
Equipment  (PM-ASE) 

•  Activity:  The  Center  provided  radiometric  equipment  and 
test  crews  to  collect  and  reduce  signature  data  on  small  arms 
(muzzle,  hardbody,  and  tracer)  and  rockets  (eject,  boost,  and 
tracer  characteristics)  on  two  separate  test  events,  SAS-W  and 
SAS-3. 

•  Benefit:  The  results  from  measured  data  will  determine 
the  variability  within  ammunition  types  and  country  of 
origin.  The  Center  will  use  the  measured  data  to  develop  the 


DOT&E  Threat  Resource  Activity-sponsored  HSIG  model, 
which  will  integrate  into  T&E  Modeling  and  Simulation 
facilities  and  support  Hostile  Fire  Detection  System  foreign 
ammunition  purchases  for  test  events. 

Navy:  Trial  Oxidizer  1 

•  Sponsor:  Naval  Research  Laboratory 

•  Activity:  The  Center  provided  radiometric  equipment  and 
test  crews  to  collect  and  reduce  signature  data  on  small  arms 
and  anti-aircraft  artillery  muzzle  flash. 

•  Benefit:  This  was  the  first  international  signature  collection 
trial  in  cooperation  with  NATO.  The  Center  will  share  the 
collected  radiometric  data  on  small  arms  and  anti-aircraft 
artillery  among  all  participants. 

NATO:  NATO  Threat  Data  Collection  Quick  Reaction 

Assessment  (QRA)  (Trial  PROTEUS) 

•  Sponsor:  NATO  Air  Capability  Group  3,  Sub-Group  2 

•  Activity:  The  Center  provided  trial  management, 
radiometric  instruments,  and  crew  during  the  collection 
of  hostile  fire  threat  signatures  at  the  Pocek  range  near 
Postojna,  Slovenia. 

•  Benefit:  This  activity  provided  a  venue  for  seven  nations 
to  collaborate  and  test  HFI  systems  for  rapid  fielding  and  to 
collect  threat  signature  data  for  use  in  developing  hostile  fire 
models. 


CM-BASED  PRE-DEPLOYMENT  TRAINING  FOR  SERVICE  MEMBER  EXERCISES 

Surface  Attack  Training  -  Nellis  AFB,  Nevada 
Angel  Thunder  -  Barry  Goldwater,  Arizona 

Texas  Air  National  Guard  Pre-Deployment  Training  -  San  Antonio,  Texas 
Mobility  Air  Force  Exercise  -  Nellis  AFB,  Nevada 
Mission  Employment  Exercise  -  Nellis  AFB,  Nevada 

Apache  Block  III  Technics,  Tactics,  and  Procedures  (TTP)  Development  Support  -  Fort  Irwin,  California 
Neptune  Falcon  -  Nellis  AFB,  Nevada 

58th  Special  Operations  Wing  Training  Support  -  Albuquerque,  New  Mexico 
4th  Battalion  501st  Aviation  Regiment  Training  Support  -  Fort  Bliss  and  Houston,  Texas 
28th  Test  and  Evaluation  Squadron  Maritime  TTP  Development  -  San  Diego,  California 
Joint  Readiness  Training  Center  Training  Support  -  Fort  Polk,  Louisiana 
509th  Weapons  Squadron  KC-135  Support  -  Roswell,  New  Mexico 

•  Sponsors:  Various 

•  Purpose:  The  Center’s  equipment  and  personnel  provided  a  simulated  threat/CM  environment  and  subject  matter  expertise  to 
observe  aircraft  sensor/ASE  systems  and  crew  reactions  to  this  environment.  Specifically,  the  Center  emphasized  simulated 
MANPADS  engagements  for  participating  aircraft.  Additionally,  the  Center  provided  MANPADS  capabilities  and  limitations 
briefings  to  pilots  and  crews  and  conducted  “hands-on”  training  at  the  end  of  the  briefings. 

•  Benefit:  Provides  realism  to  the  training  threat  environment  for  the  pilots  and  crews  to  facilitate  understanding  and  use  of  CM 
equipment,  especially  ASE.  The  Center  provided  collected  data  to  the  trainers  for  assisting  units  in  the  development/refinement 
of  TTPs  to  enhance  survivability. 
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PGW  CM  ACTIVITIES 


Navy:  Joint  Stand-Off  Weapon  (JSOW)  System 

•  Sponsors:  U.S.  Navy,  Program  Manager,  Precision  Strike 
Weapons  (PMA201) 

•  Activity:  The  Center  provided  camouflage,  concealment,  and 
deception  countermeasures  for  two  JSOW  missile  live-fire 
drops  against  stationary  land  targets.  The  missile  drops 
consisted  of  camouflage  nets  or  1R  smoke  pots  supplemented 
with  camouflage  nets. 

•  Benefit:  Conducted  regression  testing  to  determine  if  the 
JSOW  C-l  mission  capability  regarding  stationary  land  targets 
has  been  retained  in  an  1RCM  environment  given  the  recent 
addition  of  a  moving  maritime  target  capability.  The  IRCM 
land  target  test  results  will  provide  data  characterizing  imaging 
1R  seeker  performance,  which  will  be  presented  at  the  JSOW 
C-l  Operational  Test  Readiness  Review  in  1QFY13. 


National  Ground  Intelligence  Center:  Foreign 

Electro-Optical  System  (FEOS) 

•  Sponsor:  National  Ground  Intelligence  Center 

•  Activity:  The  Center  provided  a  Paveway  III  semi-active 
laser  guidance  section  and  personnel  to  collect  and  reduce 
data  showing  the  countermeasure  effects  caused  by  the 
FEOS  in  a  field  environment. 

•  Benefit:  Collected  field  test  data  on  the  FEOS  will  help 
aide  the  exploitation  efforts  and  evaluate  the  effects  on  U.S. 
domestic  guided  weapon  systems  when  subjected  to  this 
foreign  electro-optical  countermeasure  system. 


SURVIVABILITY  INITIATIVES 


HSIG  Model 

The  Center  is  leading  development  of  an  HSIG  model  to  support 
HF1  T&E  and  modeling  efforts.  The  HSIG  model  project  is 
sponsored  by  the  Threat  Resource  Activity  and  will  develop  a 
physics-based,  electro-optical  model  that  produces  signatures 
for  the  12.7  mm  Armor  Piercing  Incendiary  Tracer  round  and 
a  rocket-propelled  grenade  (RPG  7).  The  Center  completed 
development  of  the  first  small  arms  tracer  round  and  RPG 
models.  Model  validation  and  integration  to  Navy  and  Army 
facilities  will  take  place  in  FY13. 

Aircraft  Survivability  Equipment  (ASE)/Hostile  Fire  Indicator 
(HFI)  Symposium 

The  Center  held  the  fourth  ASE/HFI  symposium  and  workshop 
that  included  current  Australian,  Canadian,  New  Zealand, 

United  Kingdom,  and  U.S.  threat  detection  systems  briefings; 
“break-out”  coordination  sessions;  and  continued  development 
of  a  five-nation  methodology  for  ASE/HFI  performance  testing. 
This  Center-led  initiative  provides  a  venue  for  cross-Service 
and  international  discussion  on  the  common  problem  of 
Service  member  protection  from  threat  missile  and  ballistic 
hostile  fire  in  theater.  The  Center  has  partnered  with  the  U.S. 
Naval  Postgraduate  School  to  develop  an  academic  certificate 
of  training  to  participants  of  future  international,  classified 
ASE/HFI  T&E  Training  Symposiums  sponsored  by  the  Joint 
Countermeasures  T&E  Working  Group  (JCMT&E  WG). 

Joint  Countermeasures  T&E  Working  Group  (JCMT&E  WG) 

The  JCMT&E  WG  is  co-chartered  by  DOT&E  and 
DASD(DT&E)  to  improve  the  integration  of: 

•  Aircraft  self-protection  developments 

•  Live  weapon-fire  T&E 

•  Operational  T&E 

•  Development  of  standardized  test  methodologies 

•  Common  instrumentation  and  standards 


This  group  includes  DOT&E,  DASD(DT&E),  all  four  of  the 
U.S.  Services,  Australia,  Canada,  New  Zealand,  the  United 
Kingdom,  and  NATO  Air  Force  Armaments  Group  Sub-Group  2, 
as  members  of  a  coalition  warfare  sub-WG.  The  group  is  tasked 
with  actively  seeking  mutually  beneficial  T&E  opportunities  to 
measure  performance  and  suitability  data  necessary  to  provide 
relevant  operational  information  to  deploying  joint/coalition 
Service  members  and  for  U.S.  acquisition  decision  makers. 
Specific  efforts  included  the  following: 

•  The  JCMT&E  WG  developed,  coordinated,  and  implemented 
an  eight-year  bilateral  ASE  Cooperative  Test  and  Evaluation 
Project  Arrangement  (CTE  PA)  and  its  supporting  Project 
Management  Plan  (PMP)  with  the  United  Kingdom.  Nations’ 
defense  organizations,  ASE  Program  Offices,  DT,  OT,  and 
LFT&E  agencies  will  now  be  able  to  collaborate  on  common 
test  equipment  and  procedures,  measure  operationally  relevant 
ASE  data,  and  improve  Service  member  survivability. 

•  The  JCMT&E  WG  completed  official  negotiations  and 
concluded  agreement  of  a  bilateral  ASE  CTE  PA  and 
PMP  with  Australia  to  expand  U.S.  T&E  capabilities  and 
cooperation.  This  coordination  resulted  in  the  Center’s 
participation  in  an  Australian  hostile  fire  data  collection  trial 
that  expanded  the  U.S.  threat  database  and  will  improve  U.S. 
HFI  threat  detection  algorithms. 

•  In  support  of  High-Level  NATO  Multinational  Approaches 
Initiatives,  and  DOT&E  initiatives  to  NATO,  the  Center 
developed,  organized,  and  conducted  a  highly  successful, 
seven-nation  NATO  QRA  in  Slovenia.  U.S.  Ambassador 
Joseph  Mussomelli  praised  the  Center  for  coordination  with 
the  Embassy  and  Slovene  Forces,  and  for  its  planning  and 
execution  of  this  first  U.S. -led  NATO  QRA  in  Slovenia. 

•  The  Center  coordinated  the  first  Technical  Meeting  of  the 
Midtinational  Test  and  Evaluation  Program  memorandum 
of  understanding  in  Ottawa,  Canada,  to  initiate  negotiations 
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of  this  10-year  agreement  between  Australia,  Canada,  New 
Zealand,  the  United  Kingdom,  and  the  United  States. 

•  in  support  of  NATO  sub-group  2  and  the  Australia,  New 
Zealand,  and  U.S.  agreements,  the  JCMT&E  WG’s  efforts 
led  to  the  approval  by  the  Chief  of  the  New  Zealand  Defence 
Force  to  conduct  Trial  PAKAI  KOPERE  II  by  SG2  in  New 
Zealand  in  2QFY14.  The  sub-group  will  ask  the  Center  to 
provide  a  Multi-Spectral  Sea  and  Land  Test  Simulator,  IR 
MANPADS  Seeker  Test  Van,  and  test  personnel  to  support  this 
effort. 

Aircraft  Survivability  Equipment  (ASE)  Test  and  Evaluation 
(T&E)  Methodology  Guidebook 

The  Center  created  an  ASE  T&E  Methodology  Guidebook 
to  provide  the  DoD  with  guidance  for  planning,  executing, 
and  reporting  on  ASE  systems’  test  events.  The  ASE  systems 
addressed  in  this  guidebook  include  IRCM,  ultraviolet,  IR 
passive  missile  warning  systems,  HFI,  and  Laser  Warning 
Receiver  systems. 

Program  managers  and  T&E  leads  should  use  the  guidebook 
to  better  understand  the  process  their  teams  follow,  ensure  that 
testing  is  being  conducted  using  good  test  objectives,  and  ensure 


that  the  data  gathered  to  evaluate  those  objectives  are  valid. 

Such  a  guide  is  especially  critical  for  program  managers  and  test 
managers/leads  new  to  ASE  testing.  This  guidebook  provides 
suggested  processes  and  procedures  for  collecting  test  data,  as 
well  as  suggested  data  formats  and  data  products  for  presenting 
test  data  to  aid  the  T&E  community  in  achieving  consistency  and 
setting  expectations. 

Both  DOT&E  and  DASD(DT&E)  endorsed  the  ASE  T&E 
Methodology.  As  the  ASE  T&E  community  converges  on 
common  test  methodologies  and  approaches,  programs  can 
achieve  efficiencies  and  savings  by  utilizing  common  test  and 
range  infrastructure,  common  models  and  simulation  tools,  and 
the  ability  to  share  threat  weapon  data. 

Helicopter  Survivability  Task  Force 

The  Center  is  participating  with  the  Assistant  Secretary  of 
Defense  for  Research  and  Engineering  to  increase  aircraft 
survivability  by  coordinating  Research  and  Development 
activities  and  JCMT&E  WG  initiatives  using  tailored  projects 
for  DoD  programs  of  record  and  out-of-cycle  emergent  Service 
member  projects. 


THREAT  SIMULATOR  TEST  AND  EVALUATION  TOOLS 


The  Center,  in  conjunction  with  the  Test  Resource  Management 
Center,  is  nearing  completion  of  the  IRCM  Test  Resource 
Requirements  Study  (ITRRS)  “refresh.”  The  end  product  from 
this  effort  will  be  an  updated  roadmap  of  prioritized  projects 
necessary  to  perform  T&E  of  advanced  IRCM  and  HFI  systems. 
The  Center  completed  the  original  ITRRS  roadmap  in  2007, 
which  led  to  several  projects  being  funded  by  the  Central  Test 
and  Evaluation  Investment  Program  to  fill  the  identified  IRCM 
T&E  gaps.  Each  product  will  have  a  functional  description  of 
the  project;  the  priority  is  based  upon  Program  of  Record  test 
schedules,  requirements,  and  Service  input. 

The  Center  has  continued  to  develop  tools  for  T&E  of  IRCM 
systems  funded  by  the  USD(AT&L)  Test  Resource  Management 
Center,  Central  Test  and  Evaluation  Investment  Program. 
Currently,  the  Center  is  leading  the  development  of  the 
Multi-Spectral  Sea  and  Land  Test  Simulator  (MSALTS)  and  the 
Joint  Standard  Instrumentation  Suite  (JSIS). 

•  The  MSALTS  is  a  small,  mobile  missile  simulator  that  can 
fire  while  moving  and  simulate  all  current  tier-one  missile 
threats.  The  Center  has  designed  the  MSALTS  to  provide 
simulated  signatures  for  the  new  and  more  capable  missile 
warning  systems,  such  as  LA1RCM  Next  Generation,  DoN 
LA1RCM,  and  Joint  and  Allied  Threat  Awareness  System. 


The  Center  initiated  development  of  the  first  two  systems  in 
September  2011.  The  Center  completed  fabrication,  assembly, 
and  checkout  of  most  hardware  items  in  FY 12;  four  of  eight 
software  builds  completed  in  FY12.  Developers  plan  to 
execute  Government  acceptance  testing  of  these  MSALTS 
systems  in  September  2013. 

•  The  Center  has  completed  a  preliminary  concept  and 
development  plan  for  the  JSIS.  The  Center  intends  JSIS  to 
be  a  comprehensive,  turnkey  instrumentation  package  that 
can  be  used  during  hostile  fire  testing  and  MANPADS  firing 
events  in  and  outside  the  U.S.  to  support  model  development 
and  validation.  The  JSIS  will  provide  calibrated  signature 
measurements  for  T&E  (thus  enhancing  test  adequacy)  and 
post-test  anomaly  resolution.  The  Center  will  archive  all  the 
data  that  it  collects  using  JSIS  and  make  them  available  to  the 
Services  for  current  and  future  IRCM  programs.  The  Center 
is  actively  pursuing  JSIS  sponsorship  via  the  Central  Test  and 
Evaluation  Investment  Program.  In  the  spring  of  2012,  the 
Helicopter  Survivability  Task  Force  initiative  provided  funding 
to  develop  equipment  that  will  measure  hostile  fire  munitions’ 
time-space-position  information,  and  ultraviolet  signatures. 

The  Services  will  field  this  equipment  in  FY13,  fulfilling  an 
immediate  JSIS  development  need. 
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